Skip to content

fix(ci): stabilize PXF regression tests#127

Draft
MisterRaindrop wants to merge 21 commits into
apache:mainfrom
MisterRaindrop:fix/regression-ci
Draft

fix(ci): stabilize PXF regression tests#127
MisterRaindrop wants to merge 21 commits into
apache:mainfrom
MisterRaindrop:fix/regression-ci

Conversation

@MisterRaindrop

@MisterRaindrop MisterRaindrop commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Pin the Rocky 9 PXF service and regression shell to UTC.
  • Persist the PXF JVM timezone option and restart PXF after its CI configuration is written.
  • Allow only the large multi-block GZip and uncompressed COPY fixtures to use extended, bounded timeouts; ordinary file COPY retains the default timeout.
  • Allow the wide-row writable regression to validate its 100 MB result within a realistic CI timeout.
  • Keep BZip2 multi-block coverage with a bounded one-million-row fixture, and skip its unsupported GPDB 6 FDW write path.
  • Use the same bounded fixture size for GZip and uncompressed multi-block checks in FDW mode, while retaining the 15-million-row external-table stress coverage.
  • Disable the unavailable Rocky hpc-common repository for CI package installs; required packages come from the standard Rocky repositories.
  • Wait for at least one writable result file before validating INSERT/COPY output, preventing a pre-created empty target directory from being mistaken for a completed write.

Root cause

Parquet INT96 conversion uses the PXF JVM default timezone. Rocky 9 could start PXF with a non-UTC default, while the regression expectations use UTC. Some test groups did not restart PXF after configuration, so writing the environment file alone had no effect on the active JVM.

The hcfs jobs also exposed a separate harness limit: a 15-million-row GZip COPY crossed the former 10-minute command timeout, and a Rocky 9 uncompressed copy was truncated at the 30-minute ceiling. The respective fixtures now use bounded 30- and 60-minute limits, without extending unrelated COPY tests.

Rocky 9 additionally showed that the wide-row writable validation can exceed its former two-minute TestNG limit while copying 100 MB from HDFS. The test now has a ten-minute ceiling, while retaining an upper bound for hangs.

The former 15-million-row BZip2 fixture consistently stopped near 2.1 million rows even after fifteen minutes, while one million rows still spans many BZip2 blocks. The smaller fixture retains multi-block coverage without the unrecoverable partial write. GPDB 6 PXF FDW's native-table INSERT workaround never creates a BZip2 output directory, so that specific unsupported path is skipped. The same workaround makes the full 15-million-row GZip and uncompressed fixtures impractically slow in FDW mode, so those use the bounded fixture there; external-table coverage remains at 15 million rows.

Some Rocky 9 matrix jobs also failed before tests started because the optional hpc-common repository returned an empty mirror list. CI does not need packages from that repository, so its DNF calls now disable it explicitly.

Validation

  • bash -n ci/docker/pxf-cbdb-dev/common/script/entrypoint.sh
  • bash -n ci/docker/pxf-cbdb-dev/common/script/run_tests.sh
  • git diff --check
  • Full GitHub Actions matrix (in progress)

@MisterRaindrop MisterRaindrop changed the title fix(ci): stabilize Rocky 9 Parquet timezone fix(ci): stabilize PXF regression tests Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant