fix(ci): stabilize PXF regression tests#127
Draft
MisterRaindrop wants to merge 21 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
COPYfixtures to use extended, bounded timeouts; ordinary file COPY retains the default timeout.hpc-commonrepository for CI package installs; required packages come from the standard Rocky repositories.Root cause
Parquet INT96 conversion uses the PXF JVM default timezone. Rocky 9 could start PXF with a non-UTC default, while the regression expectations use UTC. Some test groups did not restart PXF after configuration, so writing the environment file alone had no effect on the active JVM.
The
hcfsjobs also exposed a separate harness limit: a 15-million-row GZipCOPYcrossed the former 10-minute command timeout, and a Rocky 9 uncompressed copy was truncated at the 30-minute ceiling. The respective fixtures now use bounded 30- and 60-minute limits, without extending unrelated COPY tests.Rocky 9 additionally showed that the wide-row writable validation can exceed its former two-minute TestNG limit while copying 100 MB from HDFS. The test now has a ten-minute ceiling, while retaining an upper bound for hangs.
The former 15-million-row BZip2 fixture consistently stopped near 2.1 million rows even after fifteen minutes, while one million rows still spans many BZip2 blocks. The smaller fixture retains multi-block coverage without the unrecoverable partial write. GPDB 6 PXF FDW's native-table INSERT workaround never creates a BZip2 output directory, so that specific unsupported path is skipped. The same workaround makes the full 15-million-row GZip and uncompressed fixtures impractically slow in FDW mode, so those use the bounded fixture there; external-table coverage remains at 15 million rows.
Some Rocky 9 matrix jobs also failed before tests started because the optional
hpc-commonrepository returned an empty mirror list. CI does not need packages from that repository, so its DNF calls now disable it explicitly.Validation
bash -n ci/docker/pxf-cbdb-dev/common/script/entrypoint.shbash -n ci/docker/pxf-cbdb-dev/common/script/run_tests.shgit diff --check