Gen 10 — DOM index extraction + bigger snapshot + cost cap

[drewstone]

Status: DRAFT — awaiting real-web gauntlet validation

This is the capability change Gen 9 was missing. Three coordinated changes ship together:

	change	what it solves
A	extractWithIndex action — pick-by-content over pick-by-selector	npm/mdn/python-docs failures where the LLM can't write a precise selector for data it hasn't seen
C	bigger snapshot + content line preservation (term/definition/code/pre/paragraph)	MDN/Python docs/W3C spec/arxiv pages where the data lives in <dl>/<dt>/<dd>/<code> and the old budgetSnapshot filter dropped them as "decorative"
cost cap	100K-token per-case hard cap with cost_cap_exceeded abort	the Gen 9 reddit-style death spirals (132K-173K tokens / case)

Why this is NOT another Gen 9

Gen 9 failed because it was a mechanism change without a capability change: more turns for the same LLM picking the same wrong selector. Gen 10 is a capability change — the LLM gets new information (a numbered, text-rich element index) that it didn't have before. Browser-use's per-action loop wins on the failing tasks because of exactly this mechanism.

What ships

A — extractWithIndex

• New action {action:'extractWithIndex', query:'p, dd, code', contains:'downloads'}
• Returns numbered list: [0] <p> data-testid=\"downloads\" selector: [data-testid=\"downloads\"] text: Weekly downloads: 26,543,821
• Wide query + content filter beats narrow selector. The LLM picks by index in the next turn.
• Wired into per-action loop (handler at `runner.ts`), executePlan (capture into `lastExtractOutput`, fall-through with match list), planner system prompt, validateAction parser, supervisor signature.
• Helper at `src/drivers/extract-with-index.ts` (browser-side query, hidden-element skipping, stable-selector building, 80-match cap).

C — bigger snapshot

• `budgetSnapshot` filter now preserves `term`/`definition`/`code`/`pre`/`paragraph` content lines
• Default cap raised 16k → 24k chars
• Planner cap raised 12k → 24k (planner is the most important caller for extraction tasks — it writes the runScript on first observation)
• Same-page snapshot stays at 8k (LLM has already seen the page)
• Empirical verification: Playwright DOES emit term/definition/code lines with text content. The bug was the filter, not the snapshot pipeline.

Cost cap

• `RunState.totalTokensUsed` accumulator + `tokenBudget` (default 100K, override via `Scenario.tokenBudget` or `BAD_TOKEN_BUDGET` env)
• `isTokenBudgetExhausted` checked at top of every loop iteration before next LLM call
• Returns `success: false, reason: 'cost_cap_exceeded: ...'` so the bench harness reports it cleanly
• Calibration: Gen 8 real-web ~6K, tier 1 form-multistep 60K, Gen 9 death spirals 132K-173K → 100K is well above normal max, well below death spirals

Tests

981 → 999 passing (+18 net new):

• `tests/budget-snapshot.test.ts` — 6 (filter preservation, content lines, priority bucket, paragraph handling)
• `tests/extract-with-index.test.ts` — 13 (browser-side query, contains filter, hidden element skipping, invalid selector graceful fail, stable selector, formatter, parser via Brain.parse)
• `tests/run-state.test.ts` — 7 in 'Gen 10 cost cap' describe (default, env override, accumulator, exhaustion threshold)
• `tests/runner-execute-plan.test.ts` — 2 (extractWithIndex deviation with match list, cost cap exhaustion)

Gates

• ✅ TypeScript clean (`pnpm exec tsc --noEmit`)
• ✅ Boundaries clean (`pnpm check:boundaries`)
• ✅ Full test suite (`pnpm test`) — 981/981 → growing to 999/999
• ✅ Tier1 deterministic gate PASSED
• ⏳ 3-rep real-web gauntlet — RUNNING
• ⏳ 5-rep promotion gate — gated on 3-rep wins

Lesson from Gen 9 baked in

```
Per CLAUDE.md rule #6: NO PROMOTION until 5-rep validation shows pass-rate
gain ≥+2 AND no per-task cost regression >2x AND no death-spiral runs.
```

This PR will NOT merge until the gauntlet data is in. If 3-rep doesn't show movement, the build still has value (the cost cap is a real safety net for any future iteration mechanism, the snapshot filter fix is independently useful, and extractWithIndex is a real primitive). Even in the worst case it's a strict superset of Gen 8 behavior — extractWithIndex never fires unless the LLM emits it.

What 3-rep validation will reveal

The 4 tasks I expect to move:

• mdn-array-flatmap (2/3 → 3/3): `

  ` content now visible in the snapshot AND extractWithIndex with `contains:'flatMap'` returns the signature directly

• python-docs-method-signature (2/3 → 3/3): same fix — `` content in the snapshot
• npm-package-downloads (1/3 → 2-3/3): extractWithIndex with `contains:'downloads'` reads the XHR-loaded text directly
• w3c-html-spec-find-element (3/3 → 3/3): bigger snapshot helps long-document navigation

The risk: tasks where extractWithIndex returns 80 matches and the LLM picks the wrong one.

Refs

• Pursuit doc: `.evolve/pursuits/2026-04-08-gen9-retro-and-gen10-proposal.md`
• Gen 9 retrospective: same doc
• Branch: `gen10-dom-index-extraction`