test(aws_lambda): reduce suite from 39 to 14 tests without losing coverage - #6980
Open
ericapisani wants to merge 3 commits into
Open
test(aws_lambda): reduce suite from 39 to 14 tests without losing coverage#6980ericapisani wants to merge 3 commits into
ericapisani wants to merge 3 commits into
Conversation
Reduce the docker/SAM-heavy AWS Lambda integration suite from 39 to 14 test cases (-64%) with no loss of assertions or behavior coverage, and eliminate its flakiness. Reliability fixes: - SAM template sets Architectures matching the host CPU. x86_64 containers under QEMU emulation on arm64 Macs caused 10s function timeouts, lost envelopes, and 5-9 flaky failures per run. - lambda_client fixture waits for envelope delivery to settle after each invoke instead of reading the test server race-prone. Reductions (all assertions preserved; verified by per-merge branch analysis plus a final AST-level assertion audit against the base): - Delete 3 fully redundant tests (span_origin, timeout_error, trace_continuation) whose assertions exist verbatim elsewhere - Trim equivalence-class parametrize rows (non_dict_event 7->3, headers 5->1) - Merge config-arm tests into one-test-per-feature with sequential invokes: request_data (4 arms), url_query (3 arms), user_info (2 arms), error trace context (perf on/off x new/existing), span streaming (ok/error/trace-continuation) - Factor shared helpers (_request_data_payload, _assert_segment_span_attrs) - Remove the now-unused TimeoutError lambda function Runtime: ~210s -> ~95-115s per run; 12+ consecutive green runs. Coverage of sentry_sdk/ unchanged (guard checked at every step).
Contributor
Codecov Results 📊✅ 94882 passed | ⏭️ 6343 skipped | Total: 101225 | Pass Rate: 93.73% | Execution Time: 345m 54s 📊 Comparison with Base Branch
➖ Removed Tests (1)View removed tests
All tests are passing successfully. ✅ Patch coverage is 100.00%. Project has 2511 uncovered lines. Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
+ Coverage 89.88% 89.89% +0.01%
==========================================
Files 193 193 —
Lines 24832 24832 —
Branches 8912 8912 —
==========================================
+ Hits 22319 22321 +2
- Misses 2513 2511 -2
- Partials 1416 1415 -1Generated by Codecov Action |
ericapisani
marked this pull request as ready for review
July 31, 2026 20:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #6972, applying the same reduce-tests-without-losing-coverage experiment to the AWS Lambda integration suite — the most expensive suite in the repo to run locally, since every test round-trips through dockerized AWS SAM containers.
The suite goes from 39 to 14 test cases (-64%) with no loss of assertions or behavior coverage. Reductions come from three sources:
Result: ~210s → ~115s per run on local machine, and 18 consecutive green runs (previously 5–9 flaky failures per run). Verified on py3.13 and py3.11 (full docker runs), py3.8-compatible syntax, ruff check + format clean.
Additional notes
test_timeout_errorwas dropped because its assertions are a subset oftest_timeout_error_scope_modified, which is staying. The timeout-warning behaviour is still fully covered through the latter test.However, the two used different init paths - the removed one ran through the Lambda-layer auto-init path (
init_serverless_sdk), while the one we kept uses an explicitsentry_sdk.initcall.As a result, we lose timeout-specific coverage on the layer path. However, that path is still exercised by the other tests (BasicException, InitError, BasicOk), and the warning logic is identical.
I think it's worth the trade-off here, but if there are strong opinions about adding
test_timeout_errorback in to get that timeout-specific coverage, let me know.