Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
85c675d
chore: Add branch protection
comphead Mar 2, 2026
23e7a19
[branch-53] chore: prepare 53 release (#20649)
comphead Mar 2, 2026
c466f82
[branch-53] fix: make the `sql` feature truly optional (#20625) (#20680)
comphead Mar 4, 2026
5185602
[53] fix: Fix bug in `array_has` scalar path with sliced arrays (#206…
neilconway Mar 4, 2026
b449f7a
[branch-53] fix: Return `probe_side.len()` for RightMark/Anti count(*…
jonathanc-n Mar 5, 2026
2a82d46
[branch-53] FFI_TableOptions are using default values only (#20722)
timsaucer Mar 5, 2026
f8a03d8
[branch-53] feat: parse `JsonAccess` as a binary operator, add `Opera…
Samyak2 Mar 5, 2026
3574960
[branch-53] perf: sort replace free()->try_grow() pattern with try_re…
mbutrovich Mar 6, 2026
c282508
chore(deps): pin substrait to `0.62.2` (#20827)
milenkovicm Mar 9, 2026
83c0dae
chore(deps): pin substrait version (#20848)
milenkovicm Mar 10, 2026
1196c7c
[branch-53] Fix repartition from dropping data when spilling (#20672)…
hareshkh Mar 10, 2026
7698fdc
[branch-53] fix: `HashJoin` panic with String dictionary keys (don't …
hareshkh Mar 10, 2026
ab7507f
[branch-53] cli: Fix datafusion-cli hint edge cases (#20609) (#20887)
alamb Mar 12, 2026
519866c
[branch-53] perf: Optimize `to_char` to allocate less, fix NULL handl…
alamb Mar 12, 2026
76751f1
[branch-53] fix: interval analysis error when have two filterexec tha…
alamb Mar 12, 2026
e922af9
[branch-53] correct parquet leaf index mapping when schema contains s…
alamb Mar 12, 2026
2c0a38b
[branch-53] ser/de fetch in FilterExec (#20738) (#20883)
alamb Mar 12, 2026
604bc33
[branch-53] fix: use try_shrink instead of shrink in try_resize (#204…
alamb Mar 12, 2026
129757d
[branch-53] Reattach parquet metadata cache after deserializing in da…
alamb Mar 12, 2026
e88a5fa
[branch-53] fix: do not recompute hash join exec properties if not re…
alamb Mar 12, 2026
5388c72
[branch-53] fix(spark): handle divide-by-zero in Spark `mod`/`pmod` w…
alamb Mar 12, 2026
2f2bf32
[branch-53] fix: Provide more generic API for the capacity limit pars…
alamb Mar 12, 2026
9e70cdd
[branch-53] fix: sqllogictest cannot convert <subquery> to Substrait …
alamb Mar 12, 2026
05e00ae
[branch-53] Fix DELETE/UPDATE filter extraction when predicates are p…
alamb Mar 12, 2026
19e9c06
[branch-53] fix: preserve None projection semantics across FFI bounda…
alamb Mar 12, 2026
ed25cc2
[branch-53] Fix FilterExec converting Absent column stats to Exact(NU…
alamb Mar 12, 2026
b7ccb53
[branch-53] backport: Support Spark `array_contains` builtin function…
comphead Mar 12, 2026
01437a2
[branch-53] Fix duplicate group keys after hash aggregation spill (#2…
alamb Mar 13, 2026
2c1ca2f
[branch-53] fix: SanityCheckPlan error with window functions and NVL …
alamb Mar 14, 2026
28fc91a
[branch-53] Update Release Notes (#20952)
alamb Mar 16, 2026
41b3f33
[branch-53] fix: Run release verification with --profile=ci (#20987) …
alamb Mar 17, 2026
28bb951
[branch-53] Update Cargo.lock, Fix Tokio minor breaking change (#2097…
comphead Mar 17, 2026
b83c0e0
[branch-53] fix: move overflow guard before dense ratio in hash join …
mbutrovich Mar 17, 2026
5746048
[branch-53]: fix `array_remove_*` with NULLS (#21013) (#21016)
comphead Mar 17, 2026
01df975
[branch-53] fix: InList Dictionary filter pushdown type mismatch (#20…
alamb Mar 19, 2026
21cbd38
[branch-53] Planning speed improve (port of #21084) (#21137)
blaginin Mar 25, 2026
2c3a360
[branch-53] Fix push_down_filter for children with non-empty fetch fi…
hareshkh Mar 25, 2026
c45c2ce
[branch-53] Substrait join consumer should not merge nullability of j…
hareshkh Mar 26, 2026
d24faa0
[branch-53] chore: Optimize schema rewriter usages (#21158) (#21183)
comphead Mar 26, 2026
3224e0c
[branch-53] fix: use spill writer's schema instead of the first batch…
comphead Apr 8, 2026
61d8483
[branch-53] fix: use datafusion_expr instead of datafusion crate in s…
comphead Apr 8, 2026
6fc7114
[branch-53] fix: FilterExec should drop projection when apply project…
comphead Apr 9, 2026
242fb76
[branch-53] fix: foreign inner ffi types (#21439) (#21524)
alamb Apr 9, 2026
01f2d91
[branch-53] Restore Sort unparser guard for correct ORDER BY placemen…
alamb Apr 9, 2026
637acc4
[branch-53] chore: update deps for cargo audit (#21415) (#21587)
alamb Apr 13, 2026
eae7bf4
[branch-53] Update version and add Changelog (#21559)
comphead Apr 14, 2026
42ecad5
Skip probe-side consumption when hash join build side is empty (#21068)
kosiew Apr 4, 2026
2d96876
Wire up with_new_state with DataSource (#20718)
gabotechs Mar 5, 2026
0423028
Add support for nested lists in substrait consumer (#20953)
alexanderbianchi Mar 19, 2026
53c8541
fix: string_to_array('', delim) returns empty array for PostgreSQL co…
dd-david-levin Mar 24, 2026
82e0e31
Use ListArray nullability instead of offsets for `array_element`, `ar…
tabac Apr 16, 2026
434957e
turn off submodule updating
bcmyers Mar 11, 2026
6c29a5a
feat: support `array_compact` builtin function (#21522)
comphead Apr 15, 2026
a0763db
Fix ArrayCompact incompatibility with branch-53
gabotechs Apr 21, 2026
a4a3923
Improve ergonomics for ExecutionPlanMetricsSet and MetricsSet (#21762)
gabotechs Apr 22, 2026
c7668ba
Fix custom metric display (#20643)
gabotechs Mar 3, 2026
687d16a
Add support for nested types to nullif. (#21764)
tabac Apr 28, 2026
f8c244e
Merge pull request #116 from DataDog/anastasios.bakogiannis/cherry-pi…
tabac Apr 28, 2026
5b99ba5
Add metrics for parquet sink (#20307)
xudong963 Mar 2, 2026
a2fd870
Add metric category filtering for EXPLAIN ANALYZE (#21160)
adriangb Mar 28, 2026
c252185
fix: fix elapsed_compute metric in ParquetSink to report encoding tim…
fred1268 Apr 29, 2026
0927c12
fix conflict
fred1268 Apr 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .asf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ github:
branch-52:
required_pull_request_reviews:
required_approving_review_count: 1
branch-53:
required_pull_request_reviews:
required_approving_review_count: 1
pull_requests:
# enable updating head branches of pull requests
allow_update_branch: true
Expand Down
15 changes: 15 additions & 0 deletions .github/actions/setup-submodules/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: "Setup Submodules"
description: "Initialize and update git submodules for testing"
runs:
using: "composite"
steps:
- name: Initialize and update submodules
shell: bash
run: |
git config --global --add safe.directory $GITHUB_WORKSPACE
git submodule init
# Override update=none setting for CI
git config submodule.testing.update checkout
git config submodule.parquet-testing.update checkout
git config submodule.datafusion-testing.update checkout
git submodule update --depth 1
3 changes: 2 additions & 1 deletion .github/workflows/dependencies.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,9 @@ jobs:
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/docs_pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,9 @@ jobs:
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup uv
uses: astral-sh/setup-uv@eac588ad8def6316056a12d4907a9d4d84ff7a3b # v7.3.0
- name: Install doc dependencies
Expand Down
20 changes: 12 additions & 8 deletions .github/workflows/extended.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,9 @@ jobs:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.inputs.pr_head_sha }} # will be empty if triggered by push
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Install Rust
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
Expand All @@ -101,8 +102,9 @@ jobs:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.inputs.pr_head_sha }} # will be empty if triggered by push
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
- name: Install Rust
Expand Down Expand Up @@ -148,8 +150,9 @@ jobs:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.inputs.pr_head_sha }} # will be empty if triggered by push
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand All @@ -171,12 +174,13 @@ jobs:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.inputs.pr_head_sha }} # will be empty if triggered by push
submodules: true
fetch-depth: 1
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
# Don't use setup-builder to avoid configuring RUST_BACKTRACE which is expensive
- name: Install protobuf compiler
run: |
apt-get update && apt-get install -y protobuf-compiler
- name: Run sqllogictest
run: |
cargo test --features backtrace,parquet_encryption --profile release-nonlto --test sqllogictests -- --include-sqlite
Expand Down
36 changes: 24 additions & 12 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -280,8 +280,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand Down Expand Up @@ -327,8 +328,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
run: rustup toolchain install stable
- name: Rust Dependency Cache
Expand Down Expand Up @@ -359,8 +361,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand Down Expand Up @@ -390,8 +393,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand Down Expand Up @@ -453,8 +457,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand Down Expand Up @@ -501,8 +506,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand All @@ -526,8 +532,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand Down Expand Up @@ -564,8 +571,9 @@ jobs:
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-macos-aarch64-builder
- name: Run tests (excluding doctests)
Expand Down Expand Up @@ -657,8 +665,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand All @@ -682,8 +691,9 @@ jobs:
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand All @@ -704,8 +714,9 @@ jobs:
- uses: runs-on/action@cd2b598b0515d39d78c38a02d529db87d2196d1e # v2.0.3
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
Expand Down Expand Up @@ -739,8 +750,9 @@ jobs:
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
submodules: true
fetch-depth: 1
- name: Setup Submodules
uses: ./.github/actions/setup-submodules

- name: Mark repository as safe for git
# Required for git commands inside container (avoids "dubious ownership" error)
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
[submodule "parquet-testing"]
path = parquet-testing
url = https://github.com/apache/parquet-testing.git
update = none
[submodule "testing"]
path = testing
url = https://github.com/apache/arrow-testing
update = none
[submodule "datafusion-testing"]
path = datafusion-testing
url = https://github.com/apache/datafusion-testing.git
branch = main
update = none
Loading