fix(parquet): preserve explicit mask with sparse pages

[hhhizzz]

Fully resolves the sparse-page failure mode for mask-backed Parquet row selection: explicit RowSelectionPolicy::Mask now remains correct when page pruning loads only discontiguous parts of a row group, without failing or falling back to selectors.

Related issues

• Directly resolves the sparse-page mask correctness problem in [Parquet] Support skipping pages with mask based evaluation #8845.
• Fully fixes [regression] Error with adaptive predicate pushdown: "Invalid offset in sparse column chunk data: 754, no matching page found." #9239.
• Contributes to [EPIC] Faster performance for parquet predicate evaluation for non selective filters #7456.
• The Auto policy behavior is related to [Parquet] Adaptively to pick between RowSelection and Mask filter representation #8846 and Filter pushdown selectivity threshold #9591. This PR supports Auto resolving to either mask-backed or selector-backed execution, but intentionally does not implement the broader adaptive strategy changes discussed there.

Problem and fix

Mask-backed execution previously assumed that every row covered by the absolute mask was available for decoding. Page pruning breaks this assumption: only pages containing selected rows may be loaded, leaving discontiguous ranges inside a row group. Applying the original mask as though the complete row group were present could attempt to decode a page that was never fetched and fail with invalid page offsets.

Previous handling avoided some failures by switching to selector-backed execution, which meant explicit RowSelectionPolicy::Mask was not reliable for sparse pages. This PR removes the underlying assumption and fully resolves the failure mode while preserving mask semantics.

Mask-backed execution now:

1. Determines the page-backed row ranges for every projected Parquet leaf.
2. Coalesces each leaf's ranges and intersects them across the projection.
3. Traverses only ranges that are loaded for every required leaf while retaining absolute row-group mask positions.
4. Builds BatchReadPlan::DecodeAndFilter plans carrying each decoded range and its matching MaskSlice.
5. Decodes complete loaded ranges and applies their mask slices after decoding.

The mask is not converted into selectors. For example, Page A is processed as read_records(8) -> consume_batch() -> filter_record_batch(10110101). skip_records(8) only advances across the unloaded Page B gap; Page B is never decoded.

For multi-column projections, Parquet leaves can have different page boundaries because each column chunk is encoded and paginated independently. A resulting Arrow batch needs every projected leaf for the same rows, so using the union of loaded ranges would still ask at least one leaf to read an unloaded page. The safe decode ranges are therefore the intersection across all projected leaves.

As a result:

• explicit RowSelectionPolicy::Mask remains mask-backed with sparse pages;
• mask execution never decodes missing pages;
• mask slices remain aligned across page gaps and batch boundaries;
• no fallback to selectors is required for correctness;
• Auto can safely resolve to either mask-backed or selector-backed execution.

No public APIs are changed.

Concepts introduced

• RowSelectionPolicy: the user-facing Selectors, Mask, or Auto preference.
• RowSelectionStrategy: the concrete execution strategy resolved from that preference and the selection characteristics.
• LoadedRowRanges: absolute row-group ranges backed by loaded pages for every projected leaf.
• LoadedRowRangeCursor: monotonic traversal of loaded ranges without rescanning previously visited fragments.
• BatchReadPlan: an explicit physical contract: DecodeUpTo, DecodeExact, or DecodeAndFilter.
• RowGroupReadCursor: ownership of ArrayReader execution and absolute row-group position.

These boundaries separate policy resolution, batch planning, decoder position, cardinality requirements, decoding, and post-decode filtering.

Testing

Coverage includes:

• the original explicit-mask skipped-page failure;
• explicit mask and Auto mask execution with sparse loaded pages;
• Auto resolving to selectors;
• multi-leaf range intersection with different page boundaries;
• final and nested-list projection with sparse pages;
• mask carryover across row-filter chains and batch boundaries;
• repeated planning across fragmented loaded ranges;
• exact output batch boundaries and selector short-read errors;
• loaded-range gaps, internal skips, exhaustion, and invalid ranges;
• deterministic zero-batch behavior.

Validated with:

cargo fmt --check
git diff --check
cargo test -p parquet --all-features
cargo clippy -p parquet --all-features --all-targets -- -D warnings

[hhhizzz]

@alamb 👋Hi, This PR is now ready for review and updated with the latest main.
It fully fixes the sparse-page Mask failure without falling back to selectors. Also tests on my machine. All 1,679 Parquet tests and clippy pass, and SF10 TPC-DS completed without failures or performance regression ✅
When you have time, I’d appreciate a review of the overall approach 🙏😊

[alamb]

Ok, thank ou @hhhizzz -- I will try. It just takes me a qhile to review non trivial PRs like this (not just the code, but also the approach).

[hhhizzz]

Ok, thank ou @hhhizzz -- I will try. It just takes me a qhile to review non trivial PRs like this (not just the code, but also the approach).

Thanks, and no rush at all. Please take whatever time works for you. I completely understand that reviewing the overall approach of a non-trivial change takes significantly more time than just reviewing the code.

I’ve been focusing on this area recently, and I added a few diagrams to the PR description in the hope that they make the design and execution flow easier to understand.
I’d also be very interested in taking on more maintenance responsibility in this area over time, so I’d be happy to receive any feedback on the design, implementation, tests, or how I can contribute more effectively.
I’ll try to stay involved in related community discussions and reviews, and provide code suggestions where useful. Please feel free to tag me on related issues or PRs whenever I can help.
Thanks again!