Split schema_test operations into parallel sub-tests

[Amaury Chamayou (achamayou)]

Summary

schema_test is the longest test in CI's bucket_b at ~415s. tests/schema.py already runs its sub-tests concurrently via ConcurrentRunner, but registers e2e_operations.run as a single cr.add entry that executes 29 run_* calls strictly sequentially. That one thread is the whole 415s critical path -- the other five threads all finish within 105s.

This splits those 29 calls into seven grouped entry points, registered as seven ConcurrentRunner sub-tests. No run_* function is modified, and the relative order of calls within each group is preserved, so this only changes how they are scheduled.

Measured effect

Confirmed in CI on this branch, against the median of 8 recent main runs:

Measure	main	This PR	Delta
schema_test	415.3s	107.5s	-308s (-74%)
bucket_b ctest total	836.7s	514.2s	-322s (-39%)
VMSS Virtual B wall clock	1153.5s	842s	-311s (-27%)

All six bucket_b tests pass. This also reorders the CI critical path: VMSS Virtual B was the longest job at ~1140s and is now ~842s, so VMSS Virtual C (~1044s) becomes the bound. End-to-end PR CI is ~96s shorter; the remainder is absorbed by job C.

Local runs beforehand, in an Azure Linux 3.0 container matching the CI image, showed the same effect: 402.4s before versus 115.5s / 113.9s / 114.9s over three runs, all passing, spread 1.6s.

Per-group spans measured locally after the split:

Group	Span
download-snapshot (pre-existing)	~113s
operations-snapshots	~78s
operations-persistence	~74s
operations-cose	~69s
operations-chunks	~64s
operations-config	~61s
operations-tb-snapshots	~54s
operations-offline	~29s
download	~20s
nobuiltins	~13s
tutorial, schema	~6s

schema_test is now bounded by the pre-existing download-snapshot thread rather than by operations. Splitting run_backup_snapshot_download is the obvious follow-up, but it would not shorten PR CI while job C is the critical path.

Notes

• CPU headroom is not speculative: schema_test peaked at 11 concurrent nodes before this change, while nodes_test already peaks at 22 and the recovery tests at 37 on the same runner.
• The timing-sensitive assertion in run_time_based_snapshotting (>= 8 committed snapshots over a 10s window) held up under the extra contention -- it saw 14 snapshots at 12-way concurrency versus 12 at the previous 6-way. Unchanged.
• Tests that share a workspace label are kept within a single group, so they remain sequential relative to each other. Each group gets a distinct operations-*_schema_test prefix.
• No new ctest tests are added, so tests/ci-buckets.txt is unchanged and scripts/test-buckets-checks.sh passes.
• Test infrastructure only, no user-facing change, so no CHANGELOG.md entry.

Testing

• Full CI on this branch (results above)
• schema_test three times locally, plus a pristine main baseline for comparison
• scripts/test-buckets-checks.sh
• uvx black, uvx ruff check, copyright and ASCII checks
• Verified by AST that all 29 run_* calls are preserved exactly, with no duplicates, none dropped, and relative order preserved within every group

[Copilot]

Pull request overview

Splits the long-running operations suite into parallel groups, reducing schema_test runtime while preserving each group’s test order.

Changes:

• Adds seven concurrent operations sub-tests.
• Reorganizes all 29 operations without altering their implementation.

Custom instructions used:

• .github/copilot-instructions.md

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
tests/schema.py	Registers grouped operations as concurrent sub-tests.
tests/e2e_operations.py	Defines balanced sequential entry points for each group.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.