Skip to content

chore(runner): the handsontable test discipline, ported and enforced (DEV-2203) - #220

Open
danielzytohoc wants to merge 3 commits into
feat/DEV-2203-engine-and-docs-frameworksfrom
chore/DEV-2203-test-discipline
Open

chore(runner): the handsontable test discipline, ported and enforced (DEV-2203)#220
danielzytohoc wants to merge 3 commits into
feat/DEV-2203-engine-and-docs-frameworksfrom
chore/DEV-2203-test-discipline

Conversation

@danielzytohoc

@danielzytohoc danielzytohoc commented Aug 18, 2026

Copy link
Copy Markdown

Stacked on #184 (its ci.yml is the DAG this extends).

The handsontable monorepo's test-writing skillset, ported into this repo and machine-enforced. Adapted, not copied: the runner's stack is Playwright + node --test — no Jest, no Jasmine, no frozen-suite migration story — so the rules are re-expressed in this repo's own conventions and named from this repo's own history.

What was ported, from where

Here Ported from (monorepo)
runner/docs/TESTING.md .claude/skills/test-writing-discipline/SKILL.md (the discipline core, verbatim in spirit) + handsontable/.ai/TESTING.md (the which-test-where decision rules)
.claude/skills/runner-test-discipline/ test-writing-discipline — intent-first, red-means-code-wrong-by-default, banned fake-green moves, no hollow assertions, never mock the unit under test, verify with a real run
.claude/skills/runner-playwright-e2e/ handsontable-playwright-e2e — rewired to this repo's conventions: e2e/helpers.ts instead of page objects, data-* test contracts, the env-gate taxonomy, CodeMirror/Sandpack gotchas, container-pool hygiene
runner/scripts/check-test-presence.mjs + the presence CI job .github/scripts/test-presence-gate.mjs + .ai/LOCAL-ENFORCEMENT.md (the presence gate and its escape semantics)

TESTING.md also codifies what already lived scattered in AGENTS.md and the specs — the env gates (E2E_LIVE / E2E_BASE_URL / E2E_BROKER_TOKEN / E2E_AI / E2E_STARTER_MATRIX, the latter three landing with #186/#187/#189), the house idioms (getComputedStyle not screenshots, prove-by-network-count, toPass around debounces, never read long files via .cm-content, trackSessions cleanup), and the anti-patterns with names: the hollow isMcpCreated test that asserted a copy of itself (#201), the nine days of orphaned E2E_LIVE specs no workflow ran, the failure artifact that had been empty since the workflow was added.

The presence gate

A PR that changes runner/{apps,packages,workers}/**/*.{ts,tsx} (excluding *.d.ts, *.generated.ts, public/) must also change a runner/e2e/*.spec.ts or runner/pipeline/*.test.mjs — any status: added, edited, or deleted.

Escape hatch: a trailer on any commit in the PR range, non-empty reason required. Refactor-only: <reason> — pure refactor, no behavior change. Test-plan: <reason> — the test lands in a named follow-up, or an existing spec already proves it. The gate trusts the trailer; the reviewer verifies it.

The job is PR-only (if: github.event_name == 'pull_request'): a master push arrives via workflow_call after its PR already passed the gate, and has no base to diff against anyway. No base ref resolves → clean skip, never a false block.

Housekeeping

.gitignore's blanket .claude/ becomes .claude/* + !.claude/skills/, so the skills are committed while local agent state stays ignored.

Verified

  • Six synthetic gate cases in throwaway commits: src-only fails (exit 1, ::error); src+test, src-only+Refactor-only:, src-only+Test-plan:, docs-only, and *.generated.ts-only all pass.
  • ci.yml parses (YAML).
  • pnpm test green — 409/409, nothing product-side changed.

A parallel audit is reviewing all existing tests against this ruleset; findings land as their own PRs.


Note

Low Risk
Documentation, CI policy, and a git-diff gate only—no runner app, worker, or test runtime behavior changes.

Overview
Ports the handsontable monorepo’s test-writing discipline into the runner and machine-enforces that runner source changes ship matching test changes.

runner/docs/TESTING.md becomes the canonical rules: intent-first discipline, which-test-where table, E2E env-gate taxonomy, house assertion idioms, and named anti-patterns from this repo’s history. AGENTS.md now points there instead of duplicating the full ruleset.

Two committed Claude skills (runner-test-discipline, runner-playwright-e2e) summarize discipline and Playwright mechanics; .gitignore is adjusted so .claude/skills/ can be committed while other .claude state stays ignored.

runner/scripts/check-test-presence.mjs diffs the PR against its base: changes under runner/{apps,packages,workers}/** (with exclusions) require a change to e2e/*.spec.ts or pipeline/*.test.mjs, or a Refactor-only: / Test-plan: commit trailer with a non-empty reason. A new presence job in ci.yml runs this on pull requests only (parallel to unit/build/e2e).

Reviewed by Cursor Bugbot for commit 8f2696c. Bugbot is set up for automated code reviews on this repo. Configure here.

…(DEV-2203)

Ports the monorepo's test-writing skillset into this repo, adapted to
the runner's stack (Playwright + node --test; no Jest, no Jasmine):

- runner/docs/TESTING.md — the single home for the rules: the
  discipline core (intent-first, red-means-code-wrong-by-default, the
  banned ways of faking green), the which-test-where table, the
  env-gate taxonomy (E2E_LIVE / E2E_BASE_URL / E2E_BROKER_TOKEN /
  E2E_AI / E2E_STARTER_MATRIX), the house assertion idioms, and the
  anti-patterns named from this repo's own history (#201's hollow
  guard test, the nine days of orphaned E2E_LIVE specs, the empty
  failure artifact).
- .claude/skills/runner-test-discipline + runner-playwright-e2e — the
  agent-facing entry points. .gitignore now keeps .claude/skills/
  committed while still ignoring local .claude state.
- runner/scripts/check-test-presence.mjs — the presence gate: a change
  to runner/{apps,packages,workers}/**/*.{ts,tsx} (minus *.d.ts,
  *.generated.ts, public/) must ride with a change to
  runner/e2e/*.spec.ts or runner/pipeline/*.test.mjs, unless a commit
  in the range declares 'Refactor-only: <reason>' or
  'Test-plan: <reason>'. No base ref → clean skip; a gate that
  false-blocks gets disabled, which is worse than no gate.
- ci.yml grows a presence job: PR-only (a master push arrives via
  workflow_call after its PR already passed it), parallel with unit,
  fetch-depth 0.
- runner/AGENTS.md points at TESTING.md and keeps its green-run
  bullets as the quick list.

Verified: six synthetic gate cases (src-only fails; src+test,
Refactor-only, Test-plan, docs-only, and generated-only all pass),
ci.yml parses, pnpm test green (409/409).
@danielzytohoc
danielzytohoc requested a review from demtario August 18, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants