fix: stabilize Android Maestro replay reliability

[thymikee]

Summary

Rebased the PR onto current main and updated the scope to match the real delta: Android helper default/docs clarification plus Android Maestro replay reliability fixes.

The helper-side waitForIdleTimeoutMs default stays reliability-biased at 500 for direct/common snapshots, with docs/comments clarifying that this is a maximum bounded idle wait and that callers can still pass 0 when immediate capture during ongoing animation is preferred. The persistent-session timeout budget change is not part of this PR anymore because it already landed via #796.

New runtime fixes:

• Native Maestro assertVisible now records the visible selector context, so a following tapOn can disambiguate duplicate labels on stacked/animated navigation surfaces.
• Post-gesture stabilization now takes at least two samples, so a slow first snapshot does not immediately exit stabilization with one stale sample.

Touched-file count: 7. Scope is Android snapshot helper defaults/docs and Android Maestro replay stabilization.

Validation

Rebased checks passed: focused Vitest for post-gesture stabilization and Maestro replay context (80 tests), plus pnpm check:quick.

Earlier checks for this final patch also passed before rebase: pnpm check:unit (262 unit files + 8 smoke tests), pnpm format, and pnpm build.

Live Android validation with the local built CLI passed the focused React Navigation Maestro subset covering the CI failure and flaky navigation patterns: 6/6 passed in 80.0s (auth-flow, drawer-master-detail, native-stack-card-modal, showcase-material-top-tabs, stack-card-modal, tab-view-custom-tab-bar). The previously failing native-stack-card-modal.yml passed in 13.4s.

Residual risk: the local emulator did not reproduce the GitHub Actions flake directly, so this fix is based on CI artifacts plus focused live validation rather than a before/after failing local repro.

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	1.3 MB	1.3 MB	+30 B
JS gzip	413.3 kB	413.3 kB	+29 B
npm tarball	546.4 kB	546.5 kB	+115 B
npm unpacked	1.8 MB	1.8 MB	+218 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	25.9 ms	25.5 ms	-0.4 ms
CLI --help	50.9 ms	50.5 ms	-0.4 ms

Top changed chunks:

Chunk	Raw diff	Gzip diff
dist/src/session.js	+25 B	+24 B
dist/src/2415.js	+5 B	+5 B

[thymikee]

Addressed the external review feedback.

• Rebased/cleaned the branch onto current origin/main; the two timeout-cap commits are gone from this PR because fix: relax Android snapshot helper session timeout #796 already landed that work as dd5c0be0e.
• Actual net diff is now one commit, 6c88b816c, touching 3 files: host helper default/comment, direct Java helper default, and direct helper README.
• Kept waitForIdleTimeoutMs=0 as the host default and aligned the direct Java/README default to 0 for consistency. The comment/body no longer claim daemon stabilization covers every snapshot; direct callers can still pass a positive waitForIdleTimeoutMs when they need Android's accessibility-idle gate.
• Live Android validation on emulator-5554, app org.reactnavigation.playground, passed a focused React Navigation Maestro subset:
  • drawer-master-detail.yml passed in 13.7s
  • stack-basic.yml passed in 12.5s
  • bottom-tabs.yml passed in 8.5s
• The tap-triggered navigation/assertion concern is covered by stack-basic.yml and bottom-tabs.yml; the animated drawer case is covered by drawer-master-detail.yml.
• Manual helper metadata check via open -> snapshot --json -> close showed androidSnapshot.backend=android-helper, helperTransport=persistent-session, waitForIdleTimeoutMs=0, and elapsedMs=146.
• Checks passed:
  • pnpm exec vitest run src/platforms/android/__tests__/snapshot.test.ts src/platforms/android/__tests__/snapshot-helper.test.ts src/platforms/android/__tests__/snapshot-helper-session.test.ts (70/70)
  • pnpm check:quick
  • pnpm build
  • sh ./scripts/build-android-snapshot-helper.sh 0.17.2 .tmp/android-snapshot-helper
  • ./node_modules/.bin/oxfmt --write src/platforms/android/snapshot-helper-types.ts

Residual risk: I ran a focused 3-file Maestro subset rather than the full prior 10-file subset. I did not change the real-timer/socket test inherited from #796 because that code is now on main and no longer part of this PR's diff.

[thymikee]

Additional dogfood/A-B validation after the safety question:

• Ran all 38 React Navigation Android Maestro flows individually with the PR default waitForIdleTimeoutMs=0: 38/38 passed, 416s total. Slowest flow was showcase-native-stack.yml at 24s.
• Temporarily rebuilt locally with host default waitForIdleTimeoutMs=500 and reran the same 38 flows: 38/38 passed, 419s total. Per-flow timing deltas were all under 3s; the same slowest flow was 24s.
• Manual metadata checks confirmed the A/B actually exercised the intended setting:
  • PR default: android-helper, persistent-session, waitForIdleTimeoutMs=0, elapsedMs=146
  • temporary 500 build: android-helper, persistent-session, waitForIdleTimeoutMs=500, elapsedMs=549
• Exploratory app dogfood with the PR default passed open --relaunch -> snapshot -i --json -> screenshot -> close on all installed third-party Android apps on the emulator: Agentdevicelab, Expensify dev, 2048, Neoncity, RNCLI83, Expo, and React Navigation Playground.
• Observed app states during dogfood: some RN dev builds showed redboxes and some visual-only apps/screens produced empty AX trees, but helper capture itself stayed healthy (android-helper, persistent-session, waitForIdleTimeoutMs=0) and there were no open/snapshot/screenshot/close failures.

Conclusion from this pass: I could not reproduce a reliability regression from the global 0 default in React Navigation Maestro or broad exploratory snapshots. The safety caveat remains conceptual for arbitrary tap-then-immediate-snapshot exploration, but this broader run makes the PR default look acceptable for the current Android helper path.

[thymikee]

Follow-up after the reliability discussion:

• I reverted the proposed global waitForIdleTimeoutMs=0 behavior change.
• The PR now keeps the common/helper default at 500ms and only clarifies that it is a maximum bounded wait, not a fixed sleep.
• Direct/helper callers can still pass 0 explicitly when immediate capture during ongoing animation is preferred.
• The PR title/body were updated to reflect the new scope: docs/comments only, no runtime behavior change from main.

Rationale: the A/B run did not show a React Navigation Maestro reliability problem with 0, but general agent exploration often does press/open -> immediate snapshot. Keeping the bounded 500ms default is the safer default for post-microinteraction reliability.

Validation after reverting the behavior change:

• pnpm exec vitest run src/platforms/android/__tests__/snapshot.test.ts src/platforms/android/__tests__/snapshot-helper.test.ts src/platforms/android/__tests__/snapshot-helper-session.test.ts passed 70/70.
• pnpm check:quick passed.
• sh ./scripts/build-android-snapshot-helper.sh 0.17.2 .tmp/android-snapshot-helper passed.

[thymikee]

Updated the branch for the React Navigation Android Maestro reliability follow-up.

What changed:

• Rebased the PR branch onto current main; the PR now has two commits on top of main.
• Kept the Android snapshot-helper idle wait reliability default/docs clarification from the prior PR state.
• Added a runtime fix so native Maestro assertVisible records visible context for the following tapOn, which helps disambiguate duplicate labels on stacked/animated navigation surfaces.
• Added a post-gesture stabilization guard so a slow first snapshot still gets a second sample before stabilization exits.

Validation:

• Focused Vitest after rebase: 80 tests passed for post-gesture stabilization and Maestro replay context.
• pnpm check:quick passed after rebase.
• Earlier pre-rebase checks for the same patch passed: pnpm check:unit (262 unit files + 8 smoke tests), pnpm format, and pnpm build.
• Live Android validation with the local built CLI: focused React Navigation Maestro subset passed 6/6 in 80.0s, including native-stack-card-modal.yml in 13.4s.

Residual risk: local emulator stress did not reproduce the GitHub Actions flake directly, so this is based on CI artifact analysis plus focused live validation rather than a failing local before/after repro. Expected test-time impact is minimal: the context fix is bookkeeping only; the stabilization change only adds one extra sample when the first post-gesture snapshot already exceeded the stabilization deadline.

[thymikee]

Follow-up pushed in 1d6cd203b: test --maestro discovery now avoids the generic global path sort and follows Maestro-style grouping for suite order: explicit file inputs first in caller order, directory-expanded flows next, and glob matches last. Maestro YAML/YML discovery now also runs before agent-device .ad compatibility entries in Maestro suites.\n\nValidation:\n- pnpm exec vitest run src/daemon/handlers/__tests__/session-test-discovery.test.ts\n- ./node_modules/.bin/oxfmt --write ... (direct formatter binary; pnpm format was blocked by offline pnpm signature/version switching before running oxfmt)\n- pnpm check:quick

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-06-13 17:09 UTC