branch-4.0: [fix](test) deflake several branch-4.0 P2 regression cases#64693
Merged
morningman merged 1 commit intoJun 24, 2026
Merged
Conversation
Pre-existing flaky/failing cases in the branch-4.0 P2 regression suite (surfaced by TeamCity build 198908), none caused by product changes: 1) opensky_p2 count / avgDistance / totalDistance / mostBusyOrigin: the S3 source csv.gz was actually a tar.gz, so after it was corrected the loaded row count changed (+30 rows), but the committed .out still held the old values. Backport of apache#62447 (update the four .out files and re-enable the NumberTotalRows == NumberLoadedRows load assertion). 2) segcompaction_p2/test_segcompaction_agg_keys: the AGGREGATE REPLACE winner for a duplicate key is not deterministic. S3 load parallelizes into segments whose boundaries are non-deterministic (adaptive memtable flush / memory pressure), and the read-merge keeps the first row by segment order, so which of the duplicate col_0=47 rows wins varies between runs. Backport of apache#61026 (assert exactly one row plus the value being one of the legitimate outcomes; delete the brittle .out). 3) inverted_index_p2/test_show_data (test_show_data_with_compaction): it compared index sizes with inverted_index_compaction on vs off for exact equality. For identical data the on-disk size can differ slightly due to file/merge layout, so compare within a 10% tolerance instead, which still catches gross index bloat or corruption. 4) compaction/test_base_compaction_with_dup_key_max_file_size_limit: the test builds a specific rowset layout via manual compactions and expects a manual base compaction to be rejected with E-808 (input rowset exceeds base_compaction_dup_key_max_file_size_mbytes). When the BE global disable_auto_compaction is false, background compaction races the manual steps and reshapes the rowsets, and an in-test short-circuit masked the result. Disable auto compaction cluster-wide for the duration of the test (restored in finally) and assert on the real response. 5) compaction/test_single_replica_compaction: the waitForCompaction and getTabletStatus closures assigned `code`/`out` without `def`, which the regression framework's script-source guard rejects ("defined global variables in script are not allowed: code"). apache#55933 declared only `process`; declare the remaining locals too. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QuX5zypeHzV1ZgT9o8BzoF
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
hubgeter
approved these changes
Jun 24, 2026
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Deflake several pre-existing flaky/failing cases in the branch-4.0 P2 regression suite (surfaced by an internal P2 run). None of these are caused by product behavior changes.
opensky_p2
count/avgDistance/totalDistance/mostBusyOrigin— backport of [fix](regression) fix p2 opensky failure #62447: the S3 sourcecsv.gzwas actually atar.gz, so after it was corrected the loaded row count changed (+30 rows) while the committed.outstill held the old values. Update the four.outfiles and re-enable theNumberTotalRows == NumberLoadedRowsload assertion.segcompaction_p2/test_segcompaction_agg_keys — backport of [fix](regression) Fix case test_segcompaction_agg_keys #61026: the AGGREGATE
REPLACEwinner for a duplicate key is not deterministic. S3 load parallelizes into segments whose boundaries are non-deterministic (adaptive memtable flush / memory pressure) and the read-merge keeps the first row by segment order, so which of the duplicatecol_0=47rows wins varies between runs. Assert exactly one row plus the value being one of the legitimate outcomes, and delete the brittle.out.inverted_index_p2/test_show_data (
test_show_data_with_compaction) — it compared index sizes withinverted_index_compaction_enableon vs off for exact equality. For identical data the on-disk size can differ slightly due to file/merge layout, so compare within a 10% tolerance instead, which still catches gross index bloat or corruption.compaction/test_base_compaction_with_dup_key_max_file_size_limit — the test builds a specific rowset layout via manual compactions and expects a manual base compaction to be rejected with
E-808(input rowset exceedsbase_compaction_dup_key_max_file_size_mbytes). When the BE globaldisable_auto_compactionis false, background compaction races the manual steps and reshapes the rowsets, and an in-test short-circuit masked the result. Disable auto compaction cluster-wide for the duration of the test (restored infinally) and assert on the real response.compaction/test_single_replica_compaction — the
waitForCompactionandgetTabletStatusclosures assignedcode/outwithoutdef, which the regression framework's script-source guard rejects (defined global variables in script are not allowed: code). [fix](case) fix global vars used in cases #55933 declared onlyprocess; declare the remaining locals too.Release note
None
🤖 Generated with Claude Code
https://claude.ai/code/session_01QuX5zypeHzV1ZgT9o8BzoF