Make expandable a list; default expands only DynamicTable columns#1439
Open
bendichter wants to merge 6 commits intodevfrom
Open
Make expandable a list; default expands only DynamicTable columns#1439bendichter wants to merge 6 commits intodevfrom
expandable a list; default expands only DynamicTable columns#1439bendichter wants to merge 6 commits intodevfrom
Conversation
…iers Change the default `expandable` behavior so that only VectorData (and subclasses like VectorIndex, DynamicTableRegion, EnumData) and ElementIdentifiers datasets are automatically made expandable. Other dataset types now default to contiguous (non-expandable) storage. The `expandable` parameter is changed from default=True to default=None with three-way semantics: None (default) expands only table column/id datasets, True expands all datasets, False expands none. Closes #1437 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #1439 +/- ##
==========================================
+ Coverage 93.19% 93.21% +0.01%
==========================================
Files 41 41
Lines 10062 10062
Branches 2069 2069
==========================================
+ Hits 9377 9379 +2
Misses 409 409
+ Partials 276 274 -2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
rly
reviewed
Apr 15, 2026
Also update wording in docstrings to clarify that subclasses of both VectorData and ElementIdentifiers are expanded by default, and add a roundtrip test that verifies schema-defined subclasses get maxshape set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix docstring grammar in the four expandable docval blocks. - Add a CHANGELOG entry describing the default expandable change as a bugfix. - Assert maxshape=(None,) in test_roundtrip_basic_append to explicitly cover the expandable=True path on a non-VectorData dataset. - Add an MWE test confirming docval accepts default=None with type=bool while still rejecting wrongly-typed values at runtime. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Verifies that expandable=False produces fixed-shape datasets for both VectorData columns and the ElementIdentifiers id, rather than maxshape=(None,). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
Looks good. I made some small changes: Summary
Tests added
|
oruebel
reviewed
Apr 15, 2026
…names; update related tests for new behavior
Contributor
|
Looks good to me. I'll let @rly do final approval |
…Error check - Move the six default-expandable tests from tests/unit/common/test_table.py to tests/unit/test_io_hdf5_h5tools.py; prune now-unused imports and restore PEP 8 double blank line before TestVectorIndexDtype. - Merge TestExpandableEmpty and TestExpandableCustomList into TestExpandableArg with a shared fixture, and add test_bool_raises_type_error to pin the new TypeError contract on expandable=True/False. - Strengthen TestDefaultExpandableWithReferences to assert the reference dtype and round-trip resolution of stored refs. - Update CHANGELOG to call out the change as a breaking change and note the silent default layout shift for non-DynamicTable datasets. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
expandable a list; default expands only DynamicTable columns
Contributor
|
I updated the changelog and moved the tests to a more appropriate location since expandable is no longer specific to tables. I also updated the PR summary so that the squash commit reflects the current state of changes. This is technically a breaking change and #1440 will be a major change in behavior, so the next release will be 6.0.0. |
rly
approved these changes
Apr 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
expandablebehavior inHDF5IO.write/write_builder/write_group/write_datasetso that onlyDynamicTablecolumns andidare made expandable out of the box. Datasets of all other types now default to fixed-shape on-disk layout.expandableis now a list (or tuple) of data type names with default("VectorData", "ElementIdentifiers"), not a boolean. A dataset is made expandable if its data type (or an ancestor) is in the list — so subclasses ofVectorData(e.g.DynamicTableRegion) andElementIdentifiersare also covered. Pass an empty list/tuple to disable automatic expansion entirely. PassingTrue/Falsenow raises aTypeErrorfrom docval.BuildManager.is_sub_data_type, with aget_builder_dtshort-circuit so untyped dataset builders don't hit the hierarchy lookup.Breaking changesheading with migration guidance.Closes #1437
Test plan
TestDefaultExpandable—VectorData,VectorIndex, indexedVectorData, andElementIdentifiersare expandable by defaultTestDefaultExpandableSubclasses— user-defined subclasses ofVectorData/ElementIdentifiersare expandable by default (regression test for theis_sub_data_typefix)TestDefaultExpandableWithReferences— reference-column dataset carries a reference dtype, is expandable, and refs resolve round-tripTestDefaultExpandableExplicitOverride— explicitH5DataIO(maxshape=...)is preserved, not overriddenTestExpandableArg—expandable=[]disables expansion; custom list expands only listed types;expandable=True/FalseraisesTypeErrorTestExpand.test_expand_set_shape—expandable=['QuxData']still supports the append path on non-table typesTestDTRReferences—DynamicTableRegionwith references still works🤖 Generated with Claude Code