Skip to content

Core: Preserve explicit delete_data_file() in _DeleteFiles - #3858

Open
qzyu999 wants to merge 1 commit into
apache:mainfrom
qzyu999:fix-delete-files-explicit-data-file
Open

Core: Preserve explicit delete_data_file() in _DeleteFiles#3858
qzyu999 wants to merge 1 commit into
apache:mainfrom
qzyu999:fix-delete-files-explicit-data-file

Conversation

@qzyu999

@qzyu999 qzyu999 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Closes #3857

Rationale for this change

_DeleteFiles._compute_deletes resets self._deleted_data_files = set() before scanning manifests by predicate. Files added via the inherited delete_data_file() method were silently dropped because the reset discards them before the manifest scan begins.

This means calling delete_data_file() on a _DeleteFiles instance (via update_snapshot().delete()) produces no error and no effect.

Root cause

_compute_deletes rebuilds _deleted_data_files from predicate-matched entries only. Explicit file references added before the computation were lost.

Fix

Preserve the explicit set before resetting, and include those files in the should_delete check alongside predicate evaluation. This is a 6-line production change.

Are these changes tested?

Two new tests covering:

  1. Explicit delete_data_file() on a _DeleteFiles instance deletes the file
  2. Explicit file deletion works when the table has multiple data files (only the targeted file is removed)

Both pass across all 3 catalog backends (memory, sql, sql_without_rowcount). Existing snapshot and commit-retry tests (169 total) continue to pass.

Are there any user-facing changes?

delete_data_file() on a _DeleteFiles instance now correctly deletes the specified file instead of silently doing nothing.

@qzyu999
qzyu999 force-pushed the fix-delete-files-explicit-data-file branch 2 times, most recently from 1fc26cd to b87a47c Compare August 26, 2026 04:34
…eletes

Fixes apache#3857

_compute_deletes resets self._deleted_data_files before scanning manifests by predicate. Files added via the inherited delete_data_file() method were silently dropped. Preserve them and include them in the should_delete check alongside predicate-matched files.
@qzyu999
qzyu999 force-pushed the fix-delete-files-explicit-data-file branch from b87a47c to 9813657 Compare August 26, 2026 05:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_DeleteFiles.delete_data_file() silently drops explicit file references

1 participant