Skip to content

fix(bdd): make ./scripts/run-bdd-tests.sh actually support FEATURE flag#3384

Open
chengxilo wants to merge 15 commits into
apache:masterfrom
chengxilo:seperate_bdd_test
Open

fix(bdd): make ./scripts/run-bdd-tests.sh actually support FEATURE flag#3384
chengxilo wants to merge 15 commits into
apache:masterfrom
chengxilo:seperate_bdd_test

Conversation

@chengxilo
Copy link
Copy Markdown
Contributor

@chengxilo chengxilo commented May 31, 2026

Which issue does this PR address?

Relates to #2148

Rationale

In master branch, the ./scripts/run/bdd-tests.sh doesn't support choosing a feature, while there is a FEATURE varaible.
While this script is mainly for BDD test in CI, when we implement the bdd test it is also very helpful to verify if the BDD test works. So I hope this can be improved so when I build new test scenario, it won't take me 10 minutes to build and run everything everytime I wanna test it.

What changed?

In master branch, the ./scripts/run/bdd-tests.sh doesn't support choosing a feature, while there is a FEATURE varaible. I splitted the docker-compose file for each .feature. Now when run basic_messaging scenario we only need to build and run the docker-compose.server.yml which create a single server, while in leader_redirection scenario we use docker-compose.server.yml and docker-compose.cluster.yml together. It should increase the speed to run the run-bdd-tests.sh

the clean is also updated by claude, but I think it makes sense so I kept it. Now if we use ./scripts/run/bdd-tests.sh clean, it should clean up everything that is possible to be booted but failed to be removed. I didn't actually use it tho.

And one extra change which is technically not in the scope, I removed go-race test in bdd, as I realized that it was a mistake to provide -race test in BDD test. It is only applicable for unit test with mock server.

Local Execution

  • Passed
  • Pre-commit hooks ran

AI Usage

  1. Claude Opus 4.6
  2. Code generation, I'm not good at writing shell script so it helps a lot.
  3. The code was manually modified after generation
  4. yes

@github-actions
Copy link
Copy Markdown

Thanks for the PR. It is labeled S-waiting-on-review and queued for review.

Slash commands (own line, regular comment) move it around the queue:

  • /ready - back to S-waiting-on-review after addressing feedback
  • /request-review @user-or-team - request a reviewer

See CONTRIBUTING.md for details.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label May 31, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 31, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.61%. Comparing base (a8dd3a9) to head (619a340).

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3384      +/-   ##
============================================
- Coverage     74.65%   74.61%   -0.04%     
  Complexity      943      943              
============================================
  Files          1228     1228              
  Lines        120529   120529              
  Branches      97263    97292      +29     
============================================
- Hits          89975    89928      -47     
- Misses        27614    27623       +9     
- Partials       2940     2978      +38     
Components Coverage Δ
Rust Core 75.83% <ø> (-0.02%) ⬇️
Java SDK 58.44% <ø> (ø)
C# SDK 69.41% <ø> (-0.45%) ⬇️
Python SDK 81.06% <ø> (ø)
Node SDK 91.44% <ø> (-0.10%) ⬇️
Go SDK 40.20% <ø> (ø)
see 27 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chengxilo chengxilo changed the title test: make ./scripts/run-bdd-tests.sh actually support FEATURE flag fix(bdd): make ./scripts/run-bdd-tests.sh actually support FEATURE flag May 31, 2026
Copy link
Copy Markdown
Contributor

@hubcio hubcio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one thing to confirm rather than a code issue: this PR drops the go -race BDD task (bdd-go-race, the -race flag and GO_TEST_EXTRA_FLAGS plumbing). no dangling references remain and the CI matrix entry goes away cleanly, so the removal itself is clean. worth a conscious sign-off on losing the data-race signal though - -race instruments the go test binary's own goroutines regardless of whether the server is real or mocked, so the "only applies to unit tests with a mock server" rationale isn't quite right. note coverage-baseline.yml still runs go test -race against tests/tcp_test, so some race coverage survives there.

Comment thread bdd/docker-compose.yml Outdated
Comment thread bdd/docker-compose.coverage.yml Outdated
Comment thread bdd/go/tests/suite_test.go
Comment thread bdd/docker-compose.yml Outdated
Comment thread scripts/run-bdd-tests.sh Outdated
Comment thread scripts/run-bdd-tests.sh Outdated
Comment thread bdd/docker-compose.server.yml Outdated
@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jun 1, 2026
@chengxilo
Copy link
Copy Markdown
Contributor Author

chengxilo commented Jun 1, 2026

one thing to confirm rather than a code issue: this PR drops the go -race BDD task (bdd-go-race, the -race flag and GO_TEST_EXTRA_FLAGS plumbing). no dangling references remain and the CI matrix entry goes away cleanly, so the removal itself is clean. worth a conscious sign-off on losing the data-race signal though - -race instruments the go test binary's own goroutines regardless of whether the server is real or mocked, so the "only applies to unit tests with a mock server" rationale isn't quite right. note coverage-baseline.yml still runs go test -race against tests/tcp_test, so some race coverage survives there.

Sorry it was a huge mistake. Fixed.

I removed the -race from the coverage tho, since it only only cause nagative impact and we don't need race detection for coverage report.

@chengxilo chengxilo force-pushed the seperate_bdd_test branch from 7bf8c80 to 1fb4cd8 Compare June 1, 2026 16:15
@chengxilo chengxilo force-pushed the seperate_bdd_test branch from 1fb4cd8 to 2e82506 Compare June 1, 2026 16:31
@chengxilo
Copy link
Copy Markdown
Contributor Author

/ready

@github-actions github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jun 1, 2026
@chengxilo
Copy link
Copy Markdown
Contributor Author

It's kinda funny that the CI system keeps collapse because of the device space.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review PR is waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants