-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
When performing a scan using a StreamingTable that contains sort expressions, if the projection does not include one of the sort columns it leads to an error.
PhysicalExpr Column references column 'b' at index 1 (zero-based) but input schema only has 1 columns: [\"a\"]
To Reproduce
#[tokio::test]
async fn test_sort_with_streaming_table() -> Result<()> {
let batch = record_batch!(
("a", Int32, [1, 2, 3]),
("b", Int32, [1, 2, 3])
)?;
let ctx = SessionContext::new();
let sort_order = vec![
SortExpr::new(
Expr::Column(datafusion_common::Column::new(
Option::<TableReference>::None,
"a",
)),
true,
false,
),
SortExpr::new(
Expr::Column(datafusion_common::Column::new(
Option::<TableReference>::None,
"b",
)),
true,
false,
),
];
let schema = batch.schema();
let batches = Arc::new(DummyStreamPartition {
schema: schema.clone(),
batches: vec![batch],
}) as _;
let provider = StreamingTable::try_new(schema.clone(), vec![batches])?
.with_sort_order(sort_order);
ctx.register_table("test_table", Arc::new(provider))?;
let sql = "SELECT a FROM test_table group by a";
ctx.sql(sql).await?.show().await?;
Ok(())
}
Expected behavior
The above test should pass.
Additional context
I think this can probably be resolved in StreamingTable::scan. We can project the schema before calling create_physical_sort_exprs
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working