Skip to content

Core: Discard changes when suppressing historical snapshots in CatalogHandlers - #17843

Open
ebyhr wants to merge 1 commit into
apache:mainfrom
ebyhr:ebi/fix-17538
Open

Core: Discard changes when suppressing historical snapshots in CatalogHandlers#17843
ebyhr wants to merge 1 commit into
apache:mainfrom
ebyhr:ebi/fix-17538

Conversation

@ebyhr

@ebyhr ebyhr commented Aug 27, 2026

Copy link
Copy Markdown
Member

@github-actions github-actions Bot added the core label Aug 27, 2026
.build())
.commit();

Table refsTable = catalog.loadTable(TABLE);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line throws an exception if we revert CatalogHandlers's change:

Cannot set metadata location with changes to table metadata: 1 changes
java.lang.IllegalArgumentException: Cannot set metadata location with changes to table metadata: 1 changes
	at org.apache.iceberg.rest.ErrorHandlers$DefaultErrorHandler.accept(ErrorHandlers.java:339)
	at org.apache.iceberg.rest.ErrorHandlers$TableErrorHandler.accept(ErrorHandlers.java:160)
	at org.apache.iceberg.rest.ErrorHandlers$TableErrorHandler.accept(ErrorHandlers.java:142)
	at org.apache.iceberg.rest.RESTCatalogAdapter.execute(RESTCatalogAdapter.java:649)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:732)
	at org.apache.iceberg.rest.RESTCatalogAdapter.execute(RESTCatalogAdapter.java:624)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:732)
	at org.apache.iceberg.rest.RESTCatalogAdapter.execute(RESTCatalogAdapter.java:613)
	at org.apache.iceberg.rest.BaseHTTPClient.get(BaseHTTPClient.java:101)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:732)
	at org.apache.iceberg.rest.BaseHTTPClient.get(BaseHTTPClient.java:100)
	at org.apache.iceberg.rest.RESTSessionCatalog.loadInternal(RESTSessionCatalog.java:453)
	at org.apache.iceberg.rest.RESTSessionCatalog.loadTable(RESTSessionCatalog.java:483)
	at org.apache.iceberg.catalog.BaseSessionCatalog$AsCatalog.loadTable(BaseSessionCatalog.java:105)
	at org.apache.iceberg.rest.RESTCatalog.loadTable(RESTCatalog.java:124)
	at org.apache.iceberg.rest.TestRESTCatalog.testTableSnapshotLoadingWithStatisticsOnHistoricalSnapshot(TestRESTCatalog.java:1187)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Comment thread core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java Outdated
Comment thread core/src/test/java/org/apache/iceberg/rest/TestRESTCatalog.java
@gaborkaszab

Copy link
Copy Markdown
Contributor

Thanks for opening this @ebyhr !

After I went through the description and checked how suppressHistoricalSnapshots works, I feel that the issue is not with CatalogHandlers not calling discardChanges(). What I'd like to rather scratch is if we should create any statistics removal events when in a "suppress" mode in TableMetada.rewriteSnapshotsInternal(). I don't think so. Not creating removal events in "suppress" mode (similarly as we don't create RemoveSnapshots events either) would solve the issue, right?

@uros-b

uros-b commented Aug 28, 2026

Copy link
Copy Markdown
Member

Fix looks minimal and correct to me, thank you @ebyhr and @anoopj

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Core: Discard changes when suppressing historical snapshots in CatalogHandlers

4 participants