Skip to content

ci: wire the DEV-2203 suites — post-deploy smoke, weekly canary, monthly matrix - #189

Open
danielzytohoc wants to merge 11 commits into
feat/DEV-2203-ai-live-specfrom
chore/DEV-2203-wire-workflows
Open

ci: wire the DEV-2203 suites — post-deploy smoke, weekly canary, monthly matrix#189
danielzytohoc wants to merge 11 commits into
feat/DEV-2203-ai-live-specfrom
chore/DEV-2203-wire-workflows

Conversation

@danielzytohoc

@danielzytohoc danielzytohoc commented Aug 14, 2026

Copy link
Copy Markdown

Last in the DEV-2203 stack. Stacked on #187 — merge the stack bottom-up (#181#182#183#184/#185#186#187 → this).

The coverage layers this creates

Layer Trigger Containers
PR CI (unchanged) every PR 0 — deterministic only, now incl. style-panel/version-pin/download/share-UI
Post-deploy @smoke both deploy workflows, via workflow_call 1 — one render per engine, share fixture, docs example, one Style round-trip
Weekly canary Mon 05:00 UTC, prod, AI on ~8, strictly serialized
Monthly full matrix 1st, 05:00 UTC ~50, --workers=2, unchanged discipline

e2e-live.yml

  • New dispatch inputs: ai (LLM answer checks), pkg_pr_new_ref (one real container at a pkg.pr.new build — DEV-2198 validation days).
  • Folds in the three orphaned live specs nothing ever ran: row-striping, preview-recovery's live half, import-live.
  • Steps split by what they hold: sandpack/API-read specs run parallel; container suites strictly --workers=1; the authed share step pre-checks E2E_BROKER_TOKEN against the broker and turns rot into a ::warning, not a red run.
  • Concurrency: mode-keyed as before, smoke calls keyed by run id, deployed/canary runs never cancellable (cancelled runs strand container sessions — cleanup lives in finally).

Notes

  • runner/AGENTS.md CI/CD table updated.
  • All four YAML files parse-checked. Before enabling the cron in anger: one manual dispatch against prod with everything on is the real proof — I'd do it right after the stack merges.
  • The smoke jobs add ~8–10 min after each deploy; the API deploy's smoke boots exactly the container image that deploy just rebuilt.

Note

Medium Risk
Changes deploy and scheduled CI behavior against prod and the global five-slot container pool; mis-timed concurrency or smoke gating could miss regressions or add deploy latency, but no application auth or data-path code is modified.

Overview
DEV-2203 turns the manual-only live E2E workflows into a layered schedule: PR CI stays deterministic; prod gets post-deploy @smoke, a weekly canary, and a monthly starter matrix, with container concurrency keyed so cancellations do not strand Tier-2 sessions.

e2e-live.yml now runs on workflow_dispatch (new ai and pkg_pr_new_ref inputs), weekly schedule (Mon 05:00 UTC, prod + AI), and workflow_call with smoke: true. A shared BASE_URL env resolves prod for schedules. Steps split into a fast @smoke path, parallel Sandpack/API-read specs (including previously orphaned row-striping, import-live, etc.), serialized container suites (--workers=1), optional pkg.pr.new validation, broker-token-gated authed share (warn/skip on rot, no traces), and AI live checks. Timeout rises to 90m; concurrency groups separate local, deployed, canary, and per-run smoke.

master.yml adds a smoke job that calls e2e-live.yml once after successful deploy-authoring or deploy-api (not twice on shared path changes).

e2e-starter-matrix.yml adds a monthly cron (1st at 03:00 UTC, staggered from the weekly canary) and prod fallback when schedule has no inputs.

runner/AGENTS.md CI/CD table documents the new triggers and smoke wiring.

Reviewed by Cursor Bugbot for commit 53e7a0b. Bugbot is set up for automated code reviews on this repo. Configure here.

…hly matrix

e2e-live.yml grows three ways in: the manual dispatch it always had (now
with ai and pkg_pr_new_ref inputs), a weekly Monday prod canary for
external drift (the hosted bundler, npm, the broker, docs-bucket rot —
deploy breakage is caught at deploy time, so daily would only buy
triage), and a workflow_call smoke mode both deploy workflows now invoke
after their curl checks: the @smoke subset, one container, minutes.

The full deployed run folds in the specs nothing ever ran (row-striping,
preview-recovery's live half, import-live) plus the DEV-2203 additions,
split by what they hold: the sandpack/API-read step runs parallel, the
container step strictly --workers=1 against the global pool of five, the
authed share round-trip treats a rotten E2E_BROKER_TOKEN as a notice
rather than a red run, and the AI step runs on the canary or by asking.

e2e-starter-matrix.yml gains the monthly sweep (1st, 05:00 UTC) — the
only coverage of the full starters × majors surface, at a cadence
matched to npm-release timescales. Concurrency keeps deployed runs
uncancellable so no run ever strands container sessions.
Comment thread .github/workflows/e2e-live.yml
Comment thread .github/workflows/deploy-runner-authoring.yml Outdated
Dan Zyto added 2 commits August 17, 2026 08:37
The post-deploy smoke boots a live-preview container; an unconditional
cancel-in-progress killed the whole run — smoke included — on the next
push to master, stranding that session in the global Sandbox pool
(cleanup lives in a finally a cancellation never reaches). Deploys now
queue, matching deploy-runner-api.yml.

The other #189 finding — e2e-live listing style-panel.spec.ts before it
existed on this branch — is resolved by linearising the stack: the spec
now sits in this branch's ancestry (185 → 186 → 187 → here).
Comment thread .github/workflows/e2e-live.yml
Comment thread .github/workflows/e2e-live.yml
Dan Zyto added 2 commits August 17, 2026 08:45
…ugbot #189)

The authed-share step puts a live session JWT in sessionStorage and an
Authorization header; a Playwright trace records both, GitHub does not
redact secrets inside artifact zips, and this repo is public — so a
failed canary would have published a valid @handsontable.com session
for seven days. That step now runs with --trace off --retries=0 and
scrubs test-results/ and playwright-report/ before the on-failure
upload can see them (|| status capture: Actions runs bash -e, so a
plain $? after a failing command is unreachable).

The monthly starter matrix moves to 03:00 on the 1st: at 05:00 a
Monday-the-1st would start it alongside the weekly canary, and matrix
--workers=2 plus the canary's serialized container suite against the
same global pool of five is exactly the contention both workflow
headers warn about.
…to master.yml

#184 replaced the two deploy-runner-*.yml workflows with the master.yml
orchestrator, so this branch's smoke jobs — appended to those deleted
files — become one smoke job in the orchestrator instead: the @smoke
subset runs once after whichever deploys fired, rather than once per
deploy workflow. AGENTS.md's table merges both descriptions.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ef126e9. Configure here.

Comment thread .github/workflows/e2e-live.yml
…rkflows

# Conflicts:
#	.github/workflows/e2e-live.yml
@danielzytohoc
danielzytohoc requested a review from demtario August 18, 2026 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants