Skip to content

branch-4.0: [fix](test) deflake several branch-4.0 P2 regression cases#64693

Merged
morningman merged 1 commit into
apache:branch-4.0from
morningman:fix/deflake-p2-cases-198908
Jun 24, 2026
Merged

branch-4.0: [fix](test) deflake several branch-4.0 P2 regression cases#64693
morningman merged 1 commit into
apache:branch-4.0from
morningman:fix/deflake-p2-cases-198908

Conversation

@morningman

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Deflake several pre-existing flaky/failing cases in the branch-4.0 P2 regression suite (surfaced by an internal P2 run). None of these are caused by product behavior changes.

  1. opensky_p2 count / avgDistance / totalDistance / mostBusyOrigin — backport of [fix](regression) fix p2 opensky failure #62447: the S3 source csv.gz was actually a tar.gz, so after it was corrected the loaded row count changed (+30 rows) while the committed .out still held the old values. Update the four .out files and re-enable the NumberTotalRows == NumberLoadedRows load assertion.

  2. segcompaction_p2/test_segcompaction_agg_keys — backport of [fix](regression) Fix case test_segcompaction_agg_keys #61026: the AGGREGATE REPLACE winner for a duplicate key is not deterministic. S3 load parallelizes into segments whose boundaries are non-deterministic (adaptive memtable flush / memory pressure) and the read-merge keeps the first row by segment order, so which of the duplicate col_0=47 rows wins varies between runs. Assert exactly one row plus the value being one of the legitimate outcomes, and delete the brittle .out.

  3. inverted_index_p2/test_show_data (test_show_data_with_compaction) — it compared index sizes with inverted_index_compaction_enable on vs off for exact equality. For identical data the on-disk size can differ slightly due to file/merge layout, so compare within a 10% tolerance instead, which still catches gross index bloat or corruption.

  4. compaction/test_base_compaction_with_dup_key_max_file_size_limit — the test builds a specific rowset layout via manual compactions and expects a manual base compaction to be rejected with E-808 (input rowset exceeds base_compaction_dup_key_max_file_size_mbytes). When the BE global disable_auto_compaction is false, background compaction races the manual steps and reshapes the rowsets, and an in-test short-circuit masked the result. Disable auto compaction cluster-wide for the duration of the test (restored in finally) and assert on the real response.

  5. compaction/test_single_replica_compaction — the waitForCompaction and getTabletStatus closures assigned code/out without def, which the regression framework's script-source guard rejects (defined global variables in script are not allowed: code). [fix](case) fix global vars used in cases #55933 declared only process; declare the remaining locals too.

Release note

None

🤖 Generated with Claude Code

https://claude.ai/code/session_01QuX5zypeHzV1ZgT9o8BzoF

Pre-existing flaky/failing cases in the branch-4.0 P2 regression suite
(surfaced by TeamCity build 198908), none caused by product changes:

1) opensky_p2 count / avgDistance / totalDistance / mostBusyOrigin: the
   S3 source csv.gz was actually a tar.gz, so after it was corrected the
   loaded row count changed (+30 rows), but the committed .out still held
   the old values. Backport of apache#62447 (update the four .out files and
   re-enable the NumberTotalRows == NumberLoadedRows load assertion).

2) segcompaction_p2/test_segcompaction_agg_keys: the AGGREGATE REPLACE
   winner for a duplicate key is not deterministic. S3 load parallelizes
   into segments whose boundaries are non-deterministic (adaptive memtable
   flush / memory pressure), and the read-merge keeps the first row by
   segment order, so which of the duplicate col_0=47 rows wins varies
   between runs. Backport of apache#61026 (assert exactly one row plus the value
   being one of the legitimate outcomes; delete the brittle .out).

3) inverted_index_p2/test_show_data (test_show_data_with_compaction):
   it compared index sizes with inverted_index_compaction on vs off for
   exact equality. For identical data the on-disk size can differ slightly
   due to file/merge layout, so compare within a 10% tolerance instead,
   which still catches gross index bloat or corruption.

4) compaction/test_base_compaction_with_dup_key_max_file_size_limit: the
   test builds a specific rowset layout via manual compactions and expects
   a manual base compaction to be rejected with E-808 (input rowset exceeds
   base_compaction_dup_key_max_file_size_mbytes). When the BE global
   disable_auto_compaction is false, background compaction races the manual
   steps and reshapes the rowsets, and an in-test short-circuit masked the
   result. Disable auto compaction cluster-wide for the duration of the
   test (restored in finally) and assert on the real response.

5) compaction/test_single_replica_compaction: the waitForCompaction and
   getTabletStatus closures assigned `code`/`out` without `def`, which the
   regression framework's script-source guard rejects ("defined global
   variables in script are not allowed: code"). apache#55933 declared only
   `process`; declare the remaining locals too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QuX5zypeHzV1ZgT9o8BzoF
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman morningman changed the title [fix](test) deflake several branch-4.0 P2 regression cases branch-4.0: [fix](test) deflake several branch-4.0 P2 regression cases Jun 22, 2026
@morningman

Copy link
Copy Markdown
Contributor Author

run buildall

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions Bot added approved Indicates a PR has been approved by one committer. reviewed labels Jun 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@morningman
morningman merged commit 6f1538c into apache:branch-4.0 Jun 24, 2026
30 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants