Is your feature request related to a problem or challenge?
Several methods on CatalogProvider and SchemaProvider perform potentially fallible operations but have no way to report errors:
| Trait |
Method |
Current signature |
CatalogProvider |
schema_names() |
-> Vec<String> |
CatalogProvider |
schema() |
-> Option<Arc<dyn SchemaProvider>> |
SchemaProvider |
table_names() |
-> Vec<String> |
SchemaProvider |
table_exist() |
-> bool |
For in-memory catalogs this is fine, but remote catalog implementations (e.g. paimon-rust, delta-rs Unity Catalog) need to make network
calls in these methods. When a request fails (403, timeout, connection refused, etc.), the implementation has no choice but to silently return an empty vec / None / false,
which is indistinguishable from "no schemas exist" or "table not found".
This makes debugging very difficult — a network blip looks identical to an empty catalog.
Describe the solution you'd like
Change the return types to Result<...>:
// CatalogProvider
fn schema_names(&self) -> Result<Vec<String>>;
fn schema(&self, name: &str) -> Result<Option<Arc<dyn SchemaProvider>>>;
// SchemaProvider
fn table_names(&self) -> Result<Vec<String>>;
fn table_exist(&self, name: &str) -> Result<bool>;
This is a breaking API change. All existing implementations would need to wrap their return values in Ok(...). The migration is mechanical but touches every catalog implementation.
Describe alternatives you've considered
- Logging (current workaround): Add log::error! before returning the fallback value. This surfaces the error for debugging but callers still can't react to it.
- async versions of these methods: Would also solve the problem but is a much larger change and has been intentionally avoided for planning performance reasons (as documented in CatalogProvider's doc comments).
- Keep as-is: Accept that remote catalogs must silently swallow errors in these methods.
Additional context
The other methods on these traits (table(), register_schema(), deregister_schema(), register_table(), deregister_table()) already return Result, so this would make the API consistent.
Affected files (estimated): ~21 source files + FFI boundary in datafusion-ffi.
Is your feature request related to a problem or challenge?
Several methods on
CatalogProviderandSchemaProviderperform potentially fallible operations but have no way to report errors:CatalogProviderschema_names()-> Vec<String>CatalogProviderschema()-> Option<Arc<dyn SchemaProvider>>SchemaProvidertable_names()-> Vec<String>SchemaProvidertable_exist()-> boolFor in-memory catalogs this is fine, but remote catalog implementations (e.g. paimon-rust, delta-rs Unity Catalog) need to make network
calls in these methods. When a request fails (403, timeout, connection refused, etc.), the implementation has no choice but to silently return an empty vec /
None/false,which is indistinguishable from "no schemas exist" or "table not found".
This makes debugging very difficult — a network blip looks identical to an empty catalog.
Describe the solution you'd like
Change the return types to
Result<...>:This is a breaking API change. All existing implementations would need to wrap their return values in Ok(...). The migration is mechanical but touches every catalog implementation.
Describe alternatives you've considered
Additional context
The other methods on these traits (table(), register_schema(), deregister_schema(), register_table(), deregister_table()) already return Result, so this would make the API consistent.
Affected files (estimated): ~21 source files + FFI boundary in datafusion-ffi.