chore(runner): the handsontable test discipline, ported and enforced (DEV-2203) - #220
Open
danielzytohoc wants to merge 3 commits into
Open
Conversation
…(DEV-2203) Ports the monorepo's test-writing skillset into this repo, adapted to the runner's stack (Playwright + node --test; no Jest, no Jasmine): - runner/docs/TESTING.md — the single home for the rules: the discipline core (intent-first, red-means-code-wrong-by-default, the banned ways of faking green), the which-test-where table, the env-gate taxonomy (E2E_LIVE / E2E_BASE_URL / E2E_BROKER_TOKEN / E2E_AI / E2E_STARTER_MATRIX), the house assertion idioms, and the anti-patterns named from this repo's own history (#201's hollow guard test, the nine days of orphaned E2E_LIVE specs, the empty failure artifact). - .claude/skills/runner-test-discipline + runner-playwright-e2e — the agent-facing entry points. .gitignore now keeps .claude/skills/ committed while still ignoring local .claude state. - runner/scripts/check-test-presence.mjs — the presence gate: a change to runner/{apps,packages,workers}/**/*.{ts,tsx} (minus *.d.ts, *.generated.ts, public/) must ride with a change to runner/e2e/*.spec.ts or runner/pipeline/*.test.mjs, unless a commit in the range declares 'Refactor-only: <reason>' or 'Test-plan: <reason>'. No base ref → clean skip; a gate that false-blocks gets disabled, which is worse than no gate. - ci.yml grows a presence job: PR-only (a master push arrives via workflow_call after its PR already passed it), parallel with unit, fetch-depth 0. - runner/AGENTS.md points at TESTING.md and keeps its green-run bullets as the quick list. Verified: six synthetic gate cases (src-only fails; src+test, Refactor-only, Test-plan, docs-only, and generated-only all pass), ci.yml parses, pnpm test green (409/409).
demtario
approved these changes
Aug 18, 2026
added 2 commits
August 18, 2026 17:30
…V-2203-test-discipline
…V-2203-test-discipline
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #184 (its
ci.ymlis the DAG this extends).The handsontable monorepo's test-writing skillset, ported into this repo and machine-enforced. Adapted, not copied: the runner's stack is Playwright +
node --test— no Jest, no Jasmine, no frozen-suite migration story — so the rules are re-expressed in this repo's own conventions and named from this repo's own history.What was ported, from where
runner/docs/TESTING.md.claude/skills/test-writing-discipline/SKILL.md(the discipline core, verbatim in spirit) +handsontable/.ai/TESTING.md(the which-test-where decision rules).claude/skills/runner-test-discipline/test-writing-discipline— intent-first, red-means-code-wrong-by-default, banned fake-green moves, no hollow assertions, never mock the unit under test, verify with a real run.claude/skills/runner-playwright-e2e/handsontable-playwright-e2e— rewired to this repo's conventions:e2e/helpers.tsinstead of page objects,data-*test contracts, the env-gate taxonomy, CodeMirror/Sandpack gotchas, container-pool hygienerunner/scripts/check-test-presence.mjs+ thepresenceCI job.github/scripts/test-presence-gate.mjs+.ai/LOCAL-ENFORCEMENT.md(the presence gate and its escape semantics)TESTING.mdalso codifies what already lived scattered inAGENTS.mdand the specs — the env gates (E2E_LIVE/E2E_BASE_URL/E2E_BROKER_TOKEN/E2E_AI/E2E_STARTER_MATRIX, the latter three landing with #186/#187/#189), the house idioms (getComputedStylenot screenshots, prove-by-network-count,toPassaround debounces, never read long files via.cm-content,trackSessionscleanup), and the anti-patterns with names: the hollowisMcpCreatedtest that asserted a copy of itself (#201), the nine days of orphanedE2E_LIVEspecs no workflow ran, the failure artifact that had been empty since the workflow was added.The presence gate
A PR that changes
runner/{apps,packages,workers}/**/*.{ts,tsx}(excluding*.d.ts,*.generated.ts,public/) must also change arunner/e2e/*.spec.tsorrunner/pipeline/*.test.mjs— any status: added, edited, or deleted.Escape hatch: a trailer on any commit in the PR range, non-empty reason required.
Refactor-only: <reason>— pure refactor, no behavior change.Test-plan: <reason>— the test lands in a named follow-up, or an existing spec already proves it. The gate trusts the trailer; the reviewer verifies it.The job is PR-only (
if: github.event_name == 'pull_request'): a master push arrives viaworkflow_callafter its PR already passed the gate, and has no base to diff against anyway. No base ref resolves → clean skip, never a false block.Housekeeping
.gitignore's blanket.claude/becomes.claude/*+!.claude/skills/, so the skills are committed while local agent state stays ignored.Verified
::error); src+test, src-only+Refactor-only:, src-only+Test-plan:, docs-only, and*.generated.ts-only all pass.ci.ymlparses (YAML).pnpm testgreen — 409/409, nothing product-side changed.A parallel audit is reviewing all existing tests against this ruleset; findings land as their own PRs.
Note
Low Risk
Documentation, CI policy, and a git-diff gate only—no runner app, worker, or test runtime behavior changes.
Overview
Ports the handsontable monorepo’s test-writing discipline into the runner and machine-enforces that runner source changes ship matching test changes.
runner/docs/TESTING.mdbecomes the canonical rules: intent-first discipline, which-test-where table, E2E env-gate taxonomy, house assertion idioms, and named anti-patterns from this repo’s history.AGENTS.mdnow points there instead of duplicating the full ruleset.Two committed Claude skills (
runner-test-discipline,runner-playwright-e2e) summarize discipline and Playwright mechanics;.gitignoreis adjusted so.claude/skills/can be committed while other.claudestate stays ignored.runner/scripts/check-test-presence.mjsdiffs the PR against its base: changes underrunner/{apps,packages,workers}/**(with exclusions) require a change toe2e/*.spec.tsorpipeline/*.test.mjs, or aRefactor-only:/Test-plan:commit trailer with a non-empty reason. A newpresencejob inci.ymlruns this on pull requests only (parallel to unit/build/e2e).Reviewed by Cursor Bugbot for commit 8f2696c. Bugbot is set up for automated code reviews on this repo. Configure here.