diff --git a/.agents/claims/CLAIM-ENG-RECORD-ANCHOR-RATCHET.md b/.agents/claims/CLAIM-ENG-RECORD-ANCHOR-RATCHET.md index 9a9236b13..e42414196 100644 --- a/.agents/claims/CLAIM-ENG-RECORD-ANCHOR-RATCHET.md +++ b/.agents/claims/CLAIM-ENG-RECORD-ANCHOR-RATCHET.md @@ -2,4 +2,4 @@ | Claim | Row IDs | Agent | Worktree / remote dir | Branch | Owned scope | State | Last update | |---|---|---|---|---|---|---|---| -| `CLAIM-ENG-RECORD-ANCHOR-RATCHET` | `ENG-RECORD-ANCHOR-RATCHET` (`SPIKE`) | Claude Code (opus-5), operator role — spec only; implementation goes to a fresh implementer | isolated worktree; no GPU, no build | `row/anchor-ratchet-spec`, issue [#632](https://github.com/mudler/vllm.cpp/issues/632) | Owns ONLY: NEW `.agents/specs/record-anchor-ratchet.md`; the NEW `ENG-RECORD-ANCHOR-RATCHET` row plus the summary counts it moves; and, at implementation time, the bare-citation parser, the STALE/BROKEN classifier, `scripts/record-anchor-baseline.json`, and the six RED-first cases. EXCLUDES rewriting any existing citation, and EXCLUDES every historical record (`benchmark-record.md`, `parity-ledger.md`, `state-events/`, specs recording past runs) — those cite code as it WAS and are evidence | `SPIKE` | 2026-08-14 — spec committed; implementation not started | +| `CLAIM-ENG-RECORD-ANCHOR-RATCHET` | `ENG-RECORD-ANCHOR-RATCHET` (`ACTIVE`) | Claude Code (opus-5), fresh implementer — review goes to a different agent | isolated worktree `.claude/worktrees/agent-a4c19f11b410ac0ec`; no GPU, no build | `row/anchor-ratchet-impl`, issue [#632](https://github.com/mudler/vllm.cpp/issues/632) | Owns ONLY: `.agents/specs/record-anchor-ratchet.md`; the `ENG-RECORD-ANCHOR-RATCHET` row plus the summary counts it moves; the bare-citation parser, the STALE/BROKEN classifier and the ratchet inside `scripts/check-agent-record.py`; `scripts/record-anchor-baseline.json`; the RED-first cases in `tests/scripts/test_agent_record.py`; and the preflight / CI wiring. EXCLUDES rewriting any existing citation, and EXCLUDES every historical record (`benchmark-record.md`, `parity-ledger.md`, `state-events/`, specs recording past runs) — those cite code as it WAS and are evidence | `ACTIVE` | 2026-08-18 — first fresh review returned FAIL on the recorded justification and PASS on the design, so a second fresh implementer corrected the record without redesigning anything. `origin/main` merged (161 commits, five conflicts), baseline re-derived to 39 with no new offender banked, the false "the bare form was never parsed" claim corrected on six surfaces, every ratio restated against its denominator, and #1270 found and fixed in flow. 10 cases green, 5 mutants red. Awaiting a second fresh scoped review | diff --git a/.agents/engine-matrix.md b/.agents/engine-matrix.md index 9bacf8396..75b6e9db7 100644 --- a/.agents/engine-matrix.md +++ b/.agents/engine-matrix.md @@ -44,11 +44,11 @@ forensics: roadmap_v1.md and the parity ledger. | Sampling and generation | 15 | 4 | 2 | 0 | 0 | 4 | 0 | 1 | 4 | | Structured output and tools | 7 | 0 | 4 | 0 | 0 | 2 | 0 | 0 | 1 | | Speculative decoding | 24 | 0 | 0 | 1 | 0 | 8 | 0 | 4 | 10 | -| Serving, API, CLI, library | 35 | 10 | 2 | 1 | 3 | 10 | 2 | 3 | 4 | +| Serving, API, CLI, library | 35 | 10 | 2 | 0 | 3 | 11 | 2 | 3 | 4 | | LoRA and adapters | 2 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | | Long context and attention | 10 | 5 | 0 | 0 | 1 | 0 | 1 | 0 | 3 | | Loading, tokenizer, config | 11 | 3 | 3 | 0 | 1 | 1 | 1 | 1 | 1 | -| **Total** | **164** | **35** | **17** | **5** | **12** | **37** | **8** | **11** | **38** | +| **Total** | **164** | **35** | **17** | **4** | **12** | **38** | **8** | **11** | **38** | ## Engine core and scheduling @@ -222,7 +222,7 @@ claims it. | `ENG-RELEASE-WINDOWS` | Native Windows x86_64 pre-alpha release extension: one adaptive MSVC/UCRT CPU bundle with AVX2 executed in CI and one Vulkan preview bundle, both deterministic ZIPs and authenticated by the existing release handoff | T0 | vLLM has no Windows release path; runtime behavior remains pinned to vLLM `555967922`. Platform substrate reference: llama.cpp `src/llama-mmap.cpp:520-590` @ `237ad9b961f009ae19ac29dbce4cd0c1251f94b3`; Win32 API is the OS authority | W14 Win32 portability/MSVC CPU, W15 deterministic ZIP/PE packaging + Vulkan, and W16 ten-tuple prerelease workflow/version/docs implemented for one PR | Linux portability/release mutation gates are local evidence only. Native `windows-2022` MSVC `/W4 /WX`, extracted runtime/ISA smokes, merged-SHA ten-tuple dry run, `v0.0.3-pre.1` publication, attestations, and exact 32-asset audit remain pending; no Windows ZIP exists yet | [windows-binary-release.md](specs/windows-binary-release.md); [#117](https://github.com/mudler/vllm.cpp/issues/117) | `ACTIVE` | `CLAIM-ENG-RELEASE-WINDOWS` | | `ENG-RELEASE-CONTAINERS` | Published OCI container images on GHCR, built by GitHub Actions: the same staged server bundle as `ENG-RELEASE-BINARIES`, shipped from one package `ghcr.io/mudler/vllm.cpp` with the lane in the tag — `:-cuda` / `-vulkan` / `-cpu`, the moving `:latest-cuda` / `:latest-vulkan` / `:latest-cpu`, and a bare `:latest` aliasing the cpu lane, with `ENTRYPOINT vllm-server`. Lanes `cuda` (one fat image covering every supported SM), `vulkan`, `cpu` (adaptive baseline); `rocm` blocked-preview, tracking its binary channel. Version tags are immutable; every `latest-` moves. Each lane is a `linux/amd64` + `linux/arm64` multi-arch manifest built on native runners — aarch64 is first-class here because GB10 (sm_121a), Thor (sm_110) and Orin (sm_87) are all arm64. The image contains the bundle and nothing else: no weights, no Python, no PyTorch, no compiler, no build tree. BOUNDARY: the GPU driver and container runtime stay on the host and are never bundled; Metal and MLX are NOT-CONTAINERIZABLE (no macOS container runtime and no Metal passthrough exists) and remain static-binary-only lanes, recorded as a permanent boundary rather than pending work. No image, workflow, registry package or pull is claimed to exist. | T0 | release image lanes `.buildkite/release-pipeline.yaml:34-170` and the published-image dependency boundary `docker/Dockerfile.cpu:262-290` @ `555967922` | `docker/Dockerfile` (cpu/vulkan/cuda targets calling the release scripts); `docker/healthcheck.sh`; `release/container-matrix.json`; `scripts/check-container-matrix.py`; `scripts/check-container-workflow.py`; `scripts/validate-container-image.py`; `scripts/container_tags.py`; `.github/workflows/containers.yml`; SIGTERM handler `src/vllm/entrypoints/openai/server_main.cpp` (`SignalShutdown`, all three `listen()` sites); the pre-existing `docker/Dockerfile.arm64` is an unrelated CPU bench cross-check | issues `#170`, `#312`, `#394`; `tests/scripts/test_check_container_matrix.py` 31/31; `test_check_container_workflow.py` 29/29; `test_check_cuda_fat_gencode.py` 7+4 subtests. **GB10 2026-08-11 (`promaxgb10-4ad8`, `sm_121a`, CUDA 13.3): arm64 cuda image 1.71 GB, 673/673 objects, ten-SM gencode audit PASS, and a REAL GPU boot -- `/health` 200, `/version` 200, in-container healthcheck, clean SIGTERM, `--gpus all`, host driver 580.159.03 injected.** cpu amd64 783 MB gated locally; cpu+vulkan amd64 green on hosted CI **arm64 cuda lane RUNTIME-VERIFIED on GB10 2026-08-11** -- the first accelerator-hardware evidence for any lane. Four defects were removed to get there, each found by building rather than reading: the CUDA 12.9 base could not compile `sm_110`, the BuildKit cache mount outlived its toolchain (both #366), Marlin gencode had drifted from the feature table and failed the audit on 14 correctly-compiled TUs (#394, blocking BOTH cuda tuples project-wide), and the validator could only ever produce build evidence because its boot smoke never passed `--gpus`. **NOT established: nothing is published to GHCR; amd64 cuda is unbuilt; the published arm64 image is SBSA (`targets/sbsa-linux`), so Tegra -- Thor `sm_110`, Orin `sm_87` -- is untested and NOT covered** **ORIN (Tegra) 2026-08-11: the SBSA image RUNS on Jetson AGX Orin `sm_87` (L4T R36.4.3, Docker 27.5.1) -- Qwen3-0.6B (rev `c1899de2`) loads and GENERATES via `/v1/completions`, tegrastats GR3D 95-97% during decode vs 14-15% idle.** Tegra needs `--runtime nvidia --gpus all`: `--gpus` alone is refused by the hook and `--runtime` alone mounts no driver | [container-images.md](specs/container-images.md); issues [#170](https://github.com/mudler/vllm.cpp/issues/170), [#312](https://github.com/mudler/vllm.cpp/issues/312), [#394](https://github.com/mudler/vllm.cpp/issues/394) | `ACTIVE` | `CLAIM-ENG-RELEASE-CONTAINERS-W1-W7` | | `ENG-DOCS-SITE` | Publish the 11 `docs/*.md` as a browsable GitHub Pages site at `https://mudler.github.io/vllm.cpp/` WITHOUT a second copy of the prose. A Hugo site at `website/` mounts `../docs` READ-ONLY and derives everything else from what is already in the files: each page title from the file's first `# H1`, the sidebar order from `website/data/nav.yaml`, and links through a Goldmark render hook (internal `.md` → site URL; the 139 `../.agents/**` and `../AGENTS.md` escapes → GitHub blob URLs, since the protocol tree is deliberately NOT published). **No file under `docs/` is modified, moved, renamed, or given front matter**, so `check-doc-checkpoint.py` and every protocol path reference keep working and there is no second surface that can drift — the whole point of the row. Custom lean layouts, NO theme and NO submodule: off-the-shelf docs themes read titles, weights and menus out of front matter this design deliberately does not have, so each would need its title partial, menu and link hook overridden anyway, and hugo-book additionally floors at Hugo 0.158 against the 0.146.3 pin CI and the local toolchain share. Hard prerequisite inside the repo: `classify_path` in `scripts/check-pr-size.py` FAILS CLOSED on `website/**` (verified: raises `ValueError: unclassified repository path`), so the classifier must learn the path or the PR cannot pass the project's own size gate. Hard prerequisite outside it: GitHub Pages must be enabled with the source set to GitHub Actions — the workflow is inert otherwise. A marketing landing page is explicitly OUT of scope (`README.md` stays the front door), as is any restructuring of `docs/`; the custom domain is parked behind the pending vLLM trademark question | T1 | NO vLLM analogue — upstream's docs are a separate mkdocs site and nothing in this row mirrors upstream *behavior*, so it carries no parity obligation. The STRUCTURAL reference is LocalAI's `.github/workflows/gh-pages.yml` (two Hugo sites merged into one Pages artifact), reduced to the docs half | read-only mount `website/hugo.toml:29`; title-from-H1 `website/layouts/partials/title.html:10`; link rewriting `website/layouts/_default/_markup/render-link.html:27`; guard `scripts/check-site.py:70`; deploy `.github/workflows/gh-pages.yml` | `tests/scripts/test_check_site.py:51,56,66,80,89,97` (6 mutation cases: clean tree, H1 stripped, doc absent from nav, nav entry with no file, duplicated entry, missing nav file); build evidence 14 pages with `docs/bench-evidence` + `docs/superpowers` absent from `public/` and no `href` ending in `.md`; 48 protocol links rewritten in `docs/status/`. NO published page is claimed: GitHub Pages is not yet enabled on the repository, which is the recorded stop condition holding this row at `GATING` | [gh-pages-docs-site.md](specs/gh-pages-docs-site.md); issue [#224](https://github.com/mudler/vllm.cpp/issues/224) | `READY` | `CLAIM-ENG-DOCS-SITE` | -| `ENG-RECORD-ANCHOR-RATCHET` | **The record's `path:line` citations are 82% unchecked.** `check-agent-record.py`'s `LINK_RE` (`:545`) matches only markdown links, and `local_line_anchors` (`:941`) walks only those carrying an `#L` fragment — so the dominant bare `` `file.cpp:123` `` form is never parsed: 480 link anchors are examined against **2137** invisible bare citations across the five matrices. Three gaps compound: the form is unparsed; `is_code_anchor` (`:979`) accepts a cell if **any** anchor qualifies, hiding rotted siblings; and `EVIDENCED_STATES` (`:530`) omits `ACTIVE`/`READY` entirely. Even the 18% it sees is only range-checked (`:957`), never checked to CONTAIN the symbol named beside it — every stale anchor found in the 2026-08-13/14 campaign was in range. Fix is a device-leakage-shaped ratchet over a recorded baseline, not a bulk cleanup | T1 | none — this is our own record surface; the discipline mirrors AGENTS.md §Records ("cite the `file:line` you ported from") | `scripts/check-agent-record.py:545,553,941,979,530,957` (the six sites) | none yet; six RED-first cases named in the spec, incl. the `any`-hiding case that is the shape rot used | [record-anchor-ratchet.md](specs/record-anchor-ratchet.md) | `SPIKE` | `CLAIM-ENG-RECORD-ANCHOR-RATCHET` | +| `ENG-RECORD-ANCHOR-RATCHET` | **The record's `path:line` citations were range-checked and never reported.** `check-agent-record.py` parsed BOTH forms: markdown links, and bare `` `file.cpp:123` `` through `RAW_LOCAL_ANCHOR_RE` since `ee511ca8a`. On a missing file or an out-of-range line `local_line_anchors` runs `continue`, so the bad anchor never reaches the caller, and `is_code_anchor` then answers with **any**, so one good sibling covers the rest. There was no symbol test and no report, and **32 of the 38** offenders are IN RANGE, so range-checking could not have found them. Measured at `8daa67b39`: **832 of 867** in-scope citations (**96.0%**) were already parsed and range-checked, and the **35** new to parsing sit under `.agents/`, `docs/` and `website/`; `EVIDENCED_STATES` omits `ACTIVE`/`READY` entirely and is deliberately NOT widened, because requiring an anchor there raises 85 errors across 53 rows. Even the fraction it saw was only range-checked, never checked to CONTAIN the symbol named beside it — every stale anchor found in the 2026-08-13/14 campaign was in range. LANDED as a device-leakage-shaped ratchet over a recorded baseline, never a bulk cleanup: the backlog is fixed by whoever next touches each row | T1 | none — this is our own record surface; the discipline mirrors AGENTS.md §Records ("cite the `file:line` you ported from") | parser + classifier + ratchet in `check-agent-record.py`: `BARE_CITATION_RE` `scripts/check-agent-record.py:1204` (the bare form), `cell_citations` `scripts/check-agent-record.py:1258` (both forms, with the adjacent-symbol rule), `classify_citation` `scripts/check-agent-record.py:1315` (OK / STALE / BROKEN), `RECORD_ANCHOR_STATES` `scripts/check-agent-record.py:1202` (gap 3: `ACTIVE` and `READY` join the count), `check_record_anchors` `scripts/check-agent-record.py:1438` (the two-way gate); budget in `scripts/record-anchor-baseline.json` | `RecordAnchorRatchet` `tests/scripts/test_agent_record.py:1397` — 10 cases, RED-first, including `test_one_good_link_does_not_cover_a_rotted_bare_citation` `tests/scripts/test_agent_record.py:1465`, the `any()` shape the rot hid in. Five mutants red it: report-only, `EVIDENCED_STATES` restored, links-only, first-citation-only, range-only. Measured baseline **38** (32 STALE + 6 BROKEN) over 844 OK; gate wired in `scripts/agent-preflight.sh` and the `agent-record` CI job (`--report`) | [record-anchor-ratchet.md](specs/record-anchor-ratchet.md) | `ACTIVE` | `CLAIM-ENG-RECORD-ANCHOR-RATCHET` | | `ENG-RECORD-CONFLICT-SURFACES` | Retire the shared record surfaces that make concurrent PRs conflict by construction. MEASURED at `origin/main` `d928e2c3` with `git merge-tree --write-tree` over every open PR: **16 of 29 conflict (55%), and 13 of the 16 conflict in bookkeeping files ONLY**, with no product code involved — `.agents/coordination.md` in 8, `.agents/NOW.md` in 5, `.agents/roadmap_v1.md` in 4, `scripts/check-public-doc-tables.py` in 4, `docs/STATUS.md` in 4, and any `src/`/`tests/` path in just 3. Three defects, each of which GUARANTEES rather than risks a collision. (1) `.agents/NOW.md` is a fixed-size shared buffer at EXACTLY 6000/6000 chars (`check-now-current.py:31`), so adding a row requires evicting another and every PR is a read-modify-write of one global — and the conflict is the LUCKY outcome, since a clean three-way merge would apply both evictions and both additions, silently dropping live rows and blowing the very budget the checker defends. (2) `STATUS_RATCHET = {"chars": 243245}` (`check-public-doc-tables.py:557`) is a hardcoded byte count of a DIFFERENT file that may only fall, so a PR owing `docs/STATUS.md` one lifecycle line must delete unrelated prose from another row to pay for it and edit the checker too; the checker's own comment at `:331` already records the failure (*"a ratchet pinned to the byte turns every concurrently merged row's one-line status edit into a spurious failure"*) and answered it with slack instead of removing the coupling. (3) `.agents/coordination.md`'s active-claims table is insert-at-one-anchor: the six ROCm GDN PRs (#334 #336 #341 #343 #345 #348) are ONE author's sequential stack that conflicts on nothing else, each appending a ~1,500-char row — the PR description, transcribed into a file every other claim also writes. It also contradicts the protocol it serves: `AGENTS.md` holds that *"History is git"* and *"There is no state log"*, yet both claims tables ARE state logs duplicating `gh pr list`, `row/` branch names and issue state; the argument that refuses a waiver registry applies unchanged to a claims registry. Precedent twice over — `policy.csv` retired in `0f3e44ee`, per-class line budgets retired 2026-08-10 because the gate fired on ordinary work. The exonerated surfaces share ONE property, one writer per file: `.agents/specs/.md` (one file per row, **zero conflicts** in the sample), the `*-matrix.md` inventories, and the append-only `.agents/benchmark-record.md`. SCOPE: remove `STATUS_RATCHET` and the doc-gating global counters while KEEPING the per-cell/per-paragraph caps (local, so they couple nothing); remove the active-claims table and derive claims from open PRs and branch names; drop `NOW.md`'s byte budget; order the roadmap's keyed tables by ID so distinct keys stop colliding at one anchor; and record the invariant — **no surface that every PR must write** — in `AGENTS.md`. No product source, kernel or gate semantic moves | T0 | NO vLLM analogue — this is local protocol machinery, so the mirror rule does not apply and no upstream `file:line` exists to port from. Governed instead by `AGENTS.md` §"Changing the rules or a checker", which requires a spec, a red-before test or mutation, and green-after evidence | - | - (spec-before-code: the red-before suites are named in the spec's Tests section — `tests/scripts/test_check_public_doc_tables.py`, `tests/scripts/test_check_now_current.py`, a mutation case per removed rule proving the obligation survives in the retained caps and `check-doc-checkpoint.py`, and a `git merge-tree` merge-shape regression that must be RED before the `NOW.md`/roadmap work and GREEN after) | [retire-shared-record-surfaces.md](specs/retire-shared-record-surfaces.md); issue [#364](https://github.com/mudler/vllm.cpp/issues/364) | `READY` | `CLAIM-ENG-RECORD-CONFLICT-SURFACES` | | `ENG-TRAILER-MERGE-ARTIFACTS` | The trailer gate rejects CORRECT commits because of paragraph placement, and that is why `main` is red on `agent-record`. `check-commit-trailers.py` reads trailers via `git interpret-trailers --parse`, which treats ONLY the final paragraph as the block; GitHub appends `Co-authored-by:` as a SEPARATE trailing paragraph on a squash merge, so a complete correct block becomes invisible and the gate reports it missing. MEASURED: piping `git show -s --format=%B dbd0d51c` into `git interpret-trailers --parse` prints nothing but the co-author line, and 13 of the last 30 commits on `main` fail the check -- unnoticed only because those runs were cancelled (#274), which HID the defect rather than causing it. FIX: fuse consecutive trailing TRAILER-SHAPED paragraphs before parsing. Nothing is relaxed -- the block must still exist, the marker must still sit above it, each declaration must still appear exactly once, and an AI co-author is still forbidden; the block is merely FOUND where the merge tool left it. A prose paragraph still terminates it. REJECTED IN FLIGHT and recorded because it is the more instructive half: a first attempt also collapsed identical duplicate trailers to fix the multi-commit-squash shape, which relaxes the uniqueness rule an existing test already pins. Rewriting that assertion to suit the change is what AGENTS.md forbids, and the distinction is real -- a doubled block is genuinely malformed and fixable at source, whereas the co-author case is a correct commit defeated by the parser. Reverted in full. SCOPE LIMIT, stated rather than implied: this fixes ONE of five observed shapes. `f64f2b71` (bot co-author) is a REAL violation the parse had been hiding and now correctly fails; `87308dea` (GitHub's `---------` separator), `b8293c88` (squash doubled the block) and `b580452d` (merge button, no trailers) stay red by design. Closing those is a merge-method change, not a checker change | T0 | NO vLLM analogue -- local protocol machinery, so the mirror rule does not apply and there is no upstream `file:line` to port from. Governed by `AGENTS.md` §"Changing the rules or a checker" | `scripts/check-commit-trailers.py:60` (`join_trailing_trailer_paragraphs`, `_is_trailer_paragraph`, and the fused `parsed_trailers`) | `tests/scripts/test_check_commit_trailers.py:1` 21 cases -- the RED-BEFORE appended-co-author case plus four GUARDS that keep the fusion bounded (doubled block still fails, contradictory declarations still fail, a no-trailer merge message still fails, prose after the block still fails), all four green before and after; closure [parity-ledger.md#L941](parity-ledger.md#L941) | [trailer-merge-artifacts.md](specs/trailer-merge-artifacts.md); issue [#406](https://github.com/mudler/vllm.cpp/issues/406) | `DONE` | `157080c8` | | `ENG-FORGE-COAUTHOR` | The forbidden-AI-trailer rule was catching ATTRIBUTION rather than an authorship claim, which is why bot-opened PRs red `main` on merge. GitHub composes the squash message itself and appends the account that opened the PR — `Co-authored-by: localai-org-maint-bot <...@users.noreply.github.com>` — and most PRs here are opened by a bot, so nearly every squash trips the AI-identity check. Real instance `f64f2b71`, invisible until #406 repaired the parse, which is why it reads as a new failure and is not one. The rule exists so an AI cannot claim it WROTE the code, and that stays; GitHub is recording who pressed the button, and the AI-involvement claim is already carried separately by `AI-Assisted` and `Assisted-by` in the same block. FIX: accept a `Co-authored-by` at a GitHub account noreply address even when the name matches an AI identity token, keyed on the FORGE'S OWN DOMAIN rather than the name so the exemption cannot be borrowed. A hand-written `Co-authored-by: Claude ` still fails; `Signed-off-by` is excluded from the exemption entirely, because a sign-off is a legal assertion about provenance rather than attribution. `AGENTS.md` records the same distinction in the same change so prose and checker cannot drift | T0 | NO vLLM analogue -- local protocol machinery, so the mirror rule does not apply and there is no upstream `file:line` to port from. Governed by `AGENTS.md` §"Changing the rules or a checker" | `scripts/check-commit-trailers.py:38` (`FORGE_ACCOUNT_EMAIL` and the forbidden-trailer skip) | `tests/scripts/test_check_commit_trailers.py:1` 25 cases -- the RED-BEFORE forge-bot case plus THREE guards that matter more than the relaxation because this LOOSENS a rule: a hand-written AI co-author still fails, `Signed-off-by` at the same noreply address still fails, and a human co-author still passes; all three green before and after. Real commit `f64f2b71` re-verified per commit | [forge-coauthor-attribution.md](specs/forge-coauthor-attribution.md); issue [#418](https://github.com/mudler/vllm.cpp/issues/418) | `ACTIVE` | `CLAIM-ENG-FORGE-COAUTHOR` | diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 4a46e1826..aa5205ce4 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -202,6 +202,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#834](https://github.com/mudler/vllm.cpp/issues/834) | — | No row owns router-lookahead prefetch for offloaded MoE experts. `ENG-EXPERT-STREAM` W3 copies router identifiers device to host and waits once per MoE layer (`specs/expert-streaming.md:377`), which is a synchronous stall. The only overlap work in that row is W6, and W6 runs `only if W3 trace shows wait dominance` and needs a separate accepted spike (`:380`). `ENG-WEIGHT-OFFLOAD` has a `PrefetchOffloader` arm, and it selects layers by position and never reads the router (`vllm/config/offload.py:48-76`). Prefetch is the lever that converts the per-layer fetch stall into an overlapped transfer, so the gap is recorded rather than left to be rediscovered | feature | | [#835](https://github.com/mudler/vllm.cpp/issues/835) | — | No row owns GPUDirect Storage, also called GDS or cuFile, for weight reads. `ENG-EXPERT-STREAM` W2 uses an `O_DIRECT` pool with aligned staging (`specs/expert-streaming.md:376`), which bypasses the page cache and still stages every expert through host memory. GPUDirect appears twice in the records and neither entry covers weights: `KV-MOONCAKE-STORE` names it for KV blocks over a fabric no box we own has, and `specs/lmcache-cpp-client-connector.md:305` marks GDS `NOT SCHEDULED` as an LMCache backend. The value differs by host, so a row must measure both paths before it claims a number | feature | | [#840](https://github.com/mudler/vllm.cpp/issues/840) | `POLICY-ISSUE-INTAKE` | The issue intake table sits inside `roadmap_v1.md`, which 51 of the last 60 commits touch, and two branches appending a row conflict under the default merge and merge cleanly under `merge=union`; a `.gitattributes` entry binds a path and never a section, so the table moves to `.agents/issue-index.md` and becomes append-only, and ownership becomes a network-free gate because 33 of the 185 rows name no owning row (spec [`issue-intake.md`](specs/issue-intake.md)) | bug | +| [#632](https://github.com/mudler/vllm.cpp/issues/632) | `ENG-RECORD-ANCHOR-RATCHET` | The record's own `path:line` citations were range-checked and never reported. `check-agent-record.py` parsed BOTH forms -- markdown links, and bare `file.cpp:123` through `RAW_LOCAL_ANCHOR_RE` since `ee511ca8a` -- but it dropped a failing anchor with `continue` and then answered with `any()`, so one good sibling covered the rest. There was no symbol test and no report. 832 of the 867 in-scope citations (96.0%) were already parsed, so what this row adds is the symbol test and the report rather than the parser. Every stale anchor found by hand during the 2026-08-13/14 campaign was in range. Closed by a device-leakage-shaped ratchet over `scripts/record-anchor-baseline.json`, not a bulk rewrite, spec [`record-anchor-ratchet.md`](specs/record-anchor-ratchet.md) | bug | | [#925](https://github.com/mudler/vllm.cpp/issues/925) | `MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation` | `POST /v1/audio/speech` silently ignored `audio_duration_s` — the name of the FIELD the key fills, and the spelling `speech_api.h:55` / `vllm.h:1032` / the C API all invite — so a request carrying it got a 200, a well-formed WAV, and the family's DEFAULT duration instead of the one it asked for. Every other unsupported field in `ParseSpeechRequest` is refused and named for exactly this reason; this key was the exception. It is the whole of [#852](https://github.com/mudler/vllm.cpp/issues/852): the e2e gate posted `audio_duration_s: 0.1`, ran 60 s instead (1500 AR frames not 2, 8 denoise windows not 1, 5167 vocoder latents not 6 — a ~750x job), and four runs were killed inside it and read as a hung weight load. FIXED IN FLOW: the near-miss is REFUSED, red-first in `test_speech_api.cpp` | bug | | [#775](https://github.com/mudler/vllm.cpp/issues/775) | `MODEL-TEXT-nemotron-h-nemotron-hfor-causal-lm` | `ForwardNemotronHForCausalLM` opened its type-erased handle with an unconditional `static_cast(model)`, so on any model that is not really one, every `nh.` member call was type confusion — UBSan's vptr check named `nemotron_h_registry.cpp:112:30` and `-fno-sanitize-recover=all` aborted `test_nemotron_h_scaffold`. Distinct from the TEST repair [#730](https://github.com/mudler/vllm.cpp/issues/730)/PR #784 made: that removed the stub being downcast, which cleared the symptom on the lane while leaving the cast unchecked, so the defect would have stayed invisible after the weight loader lands. FIXED by the checked `vllm::ModelAs` seam (`model_registry.h`) with the refusal authored once in `RaiseModelTypeMismatch`; spec [`nemotron-h-model.md`](specs/nemotron-h-model.md) §6d | bug | | [#847](https://github.com/mudler/vllm.cpp/issues/847) | — | Residue of the registry type-confusion class after #775: 34 `prepare`/`forward` entry points across 32 model TUs still downcast a type-erased `LoadedModel&` with an unchecked `static_cast`. The `ModelAs` seam they need already exists, so the sweep is mechanical EXCEPT for one decision it must make first — `llama_registry.cpp`, `qwen3_5_dense.cpp` and `gemma4_registry.cpp` each register THREE architectures against ONE forward, so those sites have no single architecture name to refuse under. Owed by [`nemotron-h-model.md`](specs/nemotron-h-model.md) `## Owed` until a row claims it | bug | @@ -399,6 +400,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1217](https://github.com/mudler/vllm.cpp/issues/1217) | `MODEL-NEMOTRON-H-ABI-A2P` | `ModelForwardInput::device_token_ids` carries the async runner's device-combined ids and its contract is that `token_ids` is STALE for decode rows whenever the pointer is non-null (`model_registry.h:314-324`). A registered forward that embeds the host vector then embeds the same placeholder id on EVERY decode step. The field's own comment says a model that ignores it "is simply never given one", but `runner.cpp:1408` sets the pointer for whatever model the step routes to, with no per-model opt-in and no check \| two models have now been cut from the identical divergence: Kimi-Linear (`kimi_linear_device.cpp:2270-2280`, the GB10 9/128 case) and NemotronH's paged forward under [#1157](https://github.com/mudler/vllm.cpp/issues/1157), whose A3 gate read 4/24 on GB10 against 96/96 for the same binary on CPU where the pointer is always null \| invisible because the runner sets it only under `VLLM_CPP_CUDA` with a live device mirror, so no CPU gate reaches the branch, and the failure is fluent wrong tokens rather than an error \| two closes: give `ModelFactory` an explicit `honors_device_token_ids` and have the runner fall back to the synchronous host path for a forward that has not declared it, or add a checker over the registered `.forward` entry points (a file-level grep flags ~25 false positives because several models delegate through `detail::DeviceTokenIdsScope` or the shared dense block) \| NOT fixed in the #1157 flow because one close changes a shared seam and every model factory and the other changes checker semantics, which is the "needs its own spec" case rather than the in-flow case. Listed under `## Owed` in [`nemotron-h-a2p-paged-forward.md`](specs/nemotron-h-a2p-paged-forward.md) | bug | | [#1254](https://github.com/mudler/vllm.cpp/issues/1254) | — | Once [#1202](https://github.com/mudler/vllm.cpp/issues/1202) routed the LoRA delta product through `vt::Matmul`, the majority of what a fused tensor costs is the loop that adds the delta BACK into the weight: `src/vllm/model_executor/models/ltx2_lora.cpp::Ltx2FuseLoraIntoTensor`'s bf16 branch, one thread and three out-of-line conversions per element. Measured with a same-binary A/B on the production fuser at the shipped geometry (`4096 x 450 x 4096`, 20-core Zen 5, Release, `-ffp-contract=off`, median of 5): the whole fuse is 0.1242 s, and the same call at `rank = 1` — same output size, negligible GEMM — is **0.0733 s, so the aggregator zero-fill plus the add-back loop is 59% of it** and the GEMM is ~0.046 s. Before #1202 that loop was 0.5% of the call and correctly ignored. `vt::Add` already carries the exact contract the bf16 branch needs ("computed in f32, rounded on store; `out` may alias `a`"; `include/vt/ops.h`), so the fix is the same shape #1202 took, gated the same way. The **f32 branch is NOT a match and must not be folded in with it**: it rounds the sum through bf16 before an f32 store to mirror `deltas.add_(weight)` on a bf16 aggregator followed by `.to(dtype=weight.dtype)` (`fuse_loras.py:67-68`), and `vt::Add` with an f32 output would skip that rounding and be silently more precise than the oracle — which `test_ltx2_lora`'s "the f32 target branch rounds through the bf16 accumulator" case exists to catch. Whether that branch keeps a recorded exception or the seam grows a form for it belongs in the row's spec. Owed by [`ltx25-lora-fuse-seam.md`](specs/ltx25-lora-fuse-seam.md) `## Owed` | perf | | [#1256](https://github.com/mudler/vllm.cpp/issues/1256) | `MODEL-FP8-BLOCK-LINEAR` | `main` went RED at `09597106e` (#1189 M3): `LoadQwen3_5DenseGdn` builds its `TensorExists` as `[](const std::string&) { return true; }` (`src/vllm/model_executor/models/qwen3_5_dense_weights.cpp:679`), a stub that answers YES for every name. That was harmless while the only reader of `has` was a dtype probe that went on to `get` the tensor and would throw on a name that was not there. M3's config/tensor cross-check is a different kind of reader: `IsFp8BlockProjection` asks `has(proj + ".weight_scale_inv")` and never fetches it, so on a checkpoint with no block-wise scale the stub invents one, the `!block.block_quant` guard sees tensors and config disagree, and the load is refused with a message about a tensor that does not exist. `test_qwen27_dense_forward` and `test_qwen27_dense_forward_glue_fuse_off` have been red on `main` since that commit; the scheduled CI baseline had not run past it, so the red was invisible in `main-baseline.py`. FIXED in #1189 M4's flow by asking the resolver instead of asserting: `TensorResolver` throws on a missing tensor, so a `try`/`catch` probe is the honest answer and the only one available at this seam. 9/9 cases and 583 assertions green after, 8/9 and 563 before, so the repair is measured rather than assumed. The general lesson is the one this index keeps relearning: a predicate that cannot say NO is not an instrument, and it reads as a passing probe right up until someone asks it a question whose answer matters | bug | +| [#1270](https://github.com/mudler/vllm.cpp/issues/1270) | `ENG-RECORD-ANCHOR-RATCHET` | `check-agent-record.py --write-baseline` returned the moment it had a number, which is before the `if errors:` gate at the end of `main`. A tree that failed any OTHER record check could therefore still write `scripts/record-anchor-baseline.json`, and the banked figure then carried the authority of a run that never passed, on the one file whose purpose is a number nobody may quietly raise. Found while re-deriving the baseline after merging 161 commits of `main` into #851, and FIXED in that flow: the write moves below the error gate, held by `RecordAnchorRatchet.test_a_baseline_is_never_banked_from_a_tree_with_record_errors`, captured red before the change and red again when the early return is restored | bug | | [#821](https://github.com/mudler/vllm.cpp/issues/821) | `QUANT-QWEN38-27B-GGUF-ARM` | Qwen3.8-27B's QUANTIZED ARMS had no row anywhere: the issue was referenced by three other rows and owned by none, while `BACKEND-GATE-CUDA-LLAMACPP` in the [backend matrix](backend-matrix.md) already recorded itself as blocked on it. Now three rows — this one, `QUANT-QWEN38-27B-NVFP4-ARM` and `LOAD-GGUF-MMPROJ` — because they share nothing but a model name: different file format, different loader translation unit, different oracle, different external blockers, and different tokenizers on disk. The bf16 arm of the same checkpoint is gated ([#915](https://github.com/mudler/vllm.cpp/issues/915)); `AGENTS.md` makes the quantized arms a standing requirement rather than a choice. Verified 2026-08-18 by parsing each artifact's own header, never from a repo id or a remote hash. **#821's NVFP4 pin no longer exists:** `unsloth/Qwen3.8-27B-NVFP4`@`a767244d27bd76589a3e3b2ab4e64032c4ebc7af` answers HTTP 404 and `git ls-remote` reports one ref, `refs/heads/main` = `7d6f8d4d72f56b92b3cdbf22f156b90e1bab0108`, so the user-reported `linear_attn.in_proj_qkv.input_scale` fatal is CORROBORATED at a different revision and never reproduced. At the live revision the artifact is `format: mixed-precision`, not NVFP4: an FP8 W8A8 group with **per-channel** weight scales and **dynamic per-token** activations over `self_attn.*`, `linear_attn.*`, `lm_head` and layers 56-63's MLP, an `nvfp4-pack-quantized` W4A4 group over the remaining MLP, an 8-bit static `kv_cache_scheme`, and an `ignore` list covering the vision tower — with **zero `*.input_scale` tensors in the whole checkpoint**, which is exactly why the load dies. Four independent blockers follow, and the NVFP4 half is the half closest to working. On the GGUF side the Q4_K_M file confirms #821's counts (v3, `qwen35`, 866 tensors, Q4_K/Q5_K/Q6_K/Q8_0/F32, data end == 17,106,775,008 == file size) and adds two facts it did not have: `block_count = 65` with `nextn_predict_layers = 1`, so block 64 is the MTP DRAFTER and a loader reading `block_count` as decoder depth builds the wrong graph, and `padding_token_id = 248055` against 248044 in the same model's BF16 GGUF and `null` in the official config. The `mmproj-BF16.gguf` ships BOTH halves of the temporal patch embedding (`v.patch_embd.weight` + `v.patch_embd.weight.1`), the exact thing MuseGlimmer's lacks, so it is loadable and that refusal is not precedent against it. Ten of the sixteen declared tests need no GPU and no lease. Both token gates are PENDING on named external authorities — [#857](https://github.com/mudler/vllm.cpp/issues/857) for llama.cpp gateability, [#1185](https://github.com/mudler/vllm.cpp/issues/1185) for a demonstrated vLLM model run in a lease. Spec: [`qwen38-27b-quant-arms.md`](specs/qwen38-27b-quant-arms.md) | feature | | [#1264](https://github.com/mudler/vllm.cpp/issues/1264) | `LTX25-DEVICE-RESIDENCY` | The LTX-2.5 device-residency levers were ranked by a `SPIKE` row that states in its own stop conditions that it has **no implementation authority and no fresh review** ([`ltx25-decode-speed.md`](specs/ltx25-decode-speed.md)), so nothing staged them into landable, separately reviewed work. This row takes that ownership and stages fourteen issues: [#1010](https://github.com/mudler/vllm.cpp/issues/1010) and [#1040](https://github.com/mudler/vllm.cpp/issues/1040) as W0/W1 the instrument, [#1024](https://github.com/mudler/vllm.cpp/issues/1024) re-measured in W1, [#1016](https://github.com/mudler/vllm.cpp/issues/1016)/[#1015](https://github.com/mudler/vllm.cpp/issues/1015)/[#1210](https://github.com/mudler/vllm.cpp/issues/1210) as W2, [#1021](https://github.com/mudler/vllm.cpp/issues/1021) as W3, [#1269](https://github.com/mudler/vllm.cpp/issues/1269) as W4, [#1007](https://github.com/mudler/vllm.cpp/issues/1007) with [#1011](https://github.com/mudler/vllm.cpp/issues/1011) as W5, [#1014](https://github.com/mudler/vllm.cpp/issues/1014) as W6, [#1164](https://github.com/mudler/vllm.cpp/issues/1164)'s decision point as W7, [#1012](https://github.com/mudler/vllm.cpp/issues/1012) as the parallel oracle lane, and [#1202](https://github.com/mudler/vllm.cpp/issues/1202) deferred behind W1 with its measured 2.3% attached. [#1009](https://github.com/mudler/vllm.cpp/issues/1009) is listed as CLOSED rather than staged: it landed as [#1041](https://github.com/mudler/vllm.cpp/issues/1041) and `ltx2_video_vae.cpp:170`, `:218`, `:276` now dispatch through `ParallelForRows`. **The instrument is stage zero because the ranking has three measurement defects, not because instruments are virtuous:** the evidence is unretrievable (#1040), the subject is unnamed (#1087 says *"Do not guess it from the duration"*), and the ranking is STALE — #1009 and #1208 both landed after it and neither was re-measured. W1 is an explicit gate on the order itself and the spec's `## Now` is amended with the re-derived order before W2 starts. Three corrections to the framing, each with a `file:line` at `27d5432f9`: the denoise IS device-routed (`ltx2_video.cpp:847`, `:3864-3866`) so #1024 is a symptom with more than one candidate producer and the code eliminates the denoise and, via the spike's own "no frame had been written", the decode; #1015 does NOT gate device residency because `Ltx2StreamDitToDevice` refuses `widen_to_f32` by name (`ltx2_loader.cpp:759-765`) and is a host-arm defect gating the f32 reference arm instead; and the conditioning stage is host-only BY TYPE and was in no lever table (now #1269). Spec [`ltx25-device-residency.md`](specs/ltx25-device-residency.md), row `READY` | perf | | [#1269](https://github.com/mudler/vllm.cpp/issues/1269) | `LTX25-DEVICE-RESIDENCY` | The LTX-2.5 Gemma-4 text tower is host-resident BY TYPE and invoked on a hard-coded CPU queue, so `--device cuda` never moves the conditioning pass. CPU-pinned at two independent levels at `27d5432f9`: `src/vllm/multimodal/ltx2_video.cpp:2085`, `:2799` and `:4638` each build `vt::Queue text_queue{vt::Device{vt::DeviceType::kCPU, 0}, nullptr}`, and `src/vllm/model_executor/models/ltx2_text_encoder.cpp:446` builds a SECOND hard-coded CPU queue inside the extractor, so changing the call sites alone would leave the caption projection on the host. The blocker underneath both is the type: `Ltx2GemmaTower` holds `Gemma4Weights` (`include/vllm/model_executor/models/ltx2_text_encoder.h:542-548`) of `OwnedTensor` (`include/vllm/model_executor/models/gemma4.h:114-125`) over `OwnedBytes` (`include/vllm/model_executor/models/qwen3_5_weights.h:47-56`), which has no device field; `grep -c 'Device\|kCUDA\|Queue' src/vllm/model_executor/models/gemma4_weights.cpp` returns 0 against a positive control of 16 for the identical pattern in `ltx2_device.cpp` and 4 in `ltx2_text_encoder.cpp` itself, so the zero is the finding and not a mistyped pattern. `Gemma4Model::ForwardHiddenStates` already takes a `vt::Queue` (`include/vllm/model_executor/models/gemma4.h:195`), so the forward signature is not the obstacle — a device weight arm in the shape `Ltx2StreamDitToDevice` already has for the DiT is. Cost is BOUNDED and not asserted: [#1208](https://github.com/mudler/vllm.cpp/issues/1208) measured one conditioning pass at 671.777 s of one core against 78.421 s through `vt::MatmulBT` (8.57x) on x86, a guided render pays it twice because `rows` is the constant 1024, and `docs/BENCHMARKS.md:494` records the projection as 39-100% of the ~1731 s resolution-constant phase ([#1087](https://github.com/mudler/vllm.cpp/issues/1087)) because the GB10-to-x86 per-core ratio was never measured. The Gemma-4 prefill inside the same pass has never been timed at all. Found and named by `LTX25-TEXT-LINEAR-SEAM`, whose `## Owed` says *"it needs its own issue"* and files none; filed by `LTX25-DEVICE-RESIDENCY` and staged as its W4 in [`ltx25-device-residency.md`](specs/ltx25-device-residency.md). NOT fixed in flow: it is a capability with its own correctness gate (conditioning tensors against the host arm), not a small clear fix | perf | @@ -408,6 +410,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1265](https://github.com/mudler/vllm.cpp/issues/1265) | `SGLANG-ORACLE-PERF` | `.agents/oracles/sglang.md` carried `gateable = yes` on a method the protocol now forbids. The value rests on `CLAIM-SGLANG-PERF-BENCH` (commit `0a07ac769`, 2026-07-28), which pulled and ran the `lmsysorg/sglang:v0.5.15-cu130@sha256:d0a667e` arm64 image on the GB10 — `ssh` to the host, `docker run`, and one `flock $GPU_LOCK` (`.agents/sglang-matrix.md` "Perf oracle results"; recipe `.agents/specs/cuda-sglang-low-concurrency.md:226,265,328,330`). `AGENTS.md` §"Work on a GPU happens inside a lease" now makes the `rc` lease the required path to `dgx:gpu0` and forbids reaching a fleet device by `ssh`, and it records that bypass costing a measurement on 2026-08-17 that `minimax-music3.md` §13.10 still carries as VOID. **No lease-compliant substitute is demonstrated**, verified read-only 2026-08-18: `rc run --help` has no `--image`; the `rc describe dgx:gpu0` usage sheet lists the worker as `bash`, `git`, `curl`, `wget`, `ssh`, `gcc`, `g++`, `make`, `cmake`, `ninja`, `pkg-config`, `python3`, `pip`, `venv`, `jq`, `rsync`, `tar`, `unzip`, `kubectl` and `nvidia-smi`, naming no container runtime; and `kubectl` is not the answer because a sibling pod runs outside the lease accounting and makes the fleet report the box free. `scripts/dgx-sglang-low-concurrency.sh:10-11` hard-codes an image digest and is unrunnable as written. **This retracts no measurement** — the 2026-07-28 numbers and their provenance stay in the file. It corrects a forward-looking flag that [#979](https://github.com/mudler/vllm.cpp/issues/979) and [#1255](https://github.com/mudler/vllm.cpp/issues/1255) both read when planning an SGLang arm, and it is the converse of [#1213](https://github.com/mudler/vllm.cpp/issues/1213), which corrected a record that understated a lease. FIXED IN FLOW: `gateable = no`, `evidence = #1265`, with the argument for `no` over `yes`-with-a-caveat written into the file — `scripts/check-oracle-pins.py` admits no third key, so a prose caveat leaves the machine-readable value reading `yes`. Two public documents that my own change made stale ride with it: `docs/STATUS.md` said the oracle "moves to `gateable = yes`" and `docs/BENCHMARKS.md` listed the unrun SGLang arms with no note that none is reachable. The SGLang third of the gateability debt [#647](https://github.com/mudler/vllm.cpp/issues/647) holds open therefore returns to open, on a new ground: the pin is fine, the reach is not. STILL OWED here: a demonstrated lease-compliant route or a recorded refusal naming the wall. The candidate is the PyPI wheel `sglang-0.5.15-cp312-cp312-manylinux_2_34_aarch64.whl` (sha256 `1c2d2602b4ba04c6a71d2f3bf2e3654da53987536f0d65dbe4f57cdc65c9812e`), recorded and not promised: a wheel is not the pinned image digest and needs its own asserted identity; it requires `flashinfer_python[cu13]==0.6.12`, a `py3-none-any` wheel that compiles at run time and so needs the toolkit and headers [#1253](https://github.com/mudler/vllm.cpp/issues/1253) records as absent or partial on this worker; and `sglang_kernel-0.4.4`'s aarch64 wheel is 34,243,333 bytes against 615,071,908 for x86_64, so its `sm_121a` coverage is unestablished | record | | [#1262](https://github.com/mudler/vllm.cpp/issues/1262) | `FIX-TRAILER-LANE-CUTOVER` | `281b4bc76c0e` is on `main` carrying `Assisted-by: AGENT:claude-opus-5 CLI`, which is missing the bracketed `[TOOL]` the grammar requires. Measured with the checker itself from `origin/main` `27d5432f9`: `--range '281b4bc76~1..origin/main'` is `rc=1` with that one offender and `--range '281b4bc76..origin/main'` is `rc=0`, so the blast radius is one commit carrying one error. It CANNOT be repaired, because correcting a landed message rewrites `main`, and it does NOT clear itself: the main lane walks `LAST_GREEN..head` and `LAST_GREEN` advances only on a green run, so every later push re-walks a range that still contains it -- the same property `ci.yml:74` relies on to make a cancelled run lossless makes an unrepairable red permanent. The branch commits were all correct; `squash_merge_commit_message = PR_BODY` landed the pull request body, which still held the pre-repair value, and the guard that reads the body (`ci.yml:626-635`, #848) was `pending` at merge time because the runner pool was saturated -- it did not fail, it never ran. FIXED by an enumerated exception keyed on the full commit oid AND the exact rendered error string, printed on every run that applies it. `--cutover` was measured and REJECTED as the instrument: `--cutover 281b4bc76` does not excuse `281b4bc76` at all, because `merge-base --is-ancestor X X` succeeds and the cutover commit is checked strictly; naming its child does excuse it and drops 2986 ancestors to the marker-only check, waiving defects nobody has read; and it is a value that can be moved to hide the next red, which is the failure mode `AGENTS.md` §"Changing the rules or a checker" exists to prevent. Spec [`fix-trailer-lane-cutover.md`](specs/fix-trailer-lane-cutover.md) | bug | | [#1263](https://github.com/mudler/vllm.cpp/issues/1263) | — | No operator-side check reads the pull request BODY before a squash merge. The CI guard that does (`ci.yml:626-635`, #848) runs the same checker over `$PR_BODY` with `--filled`, but it is not a precondition of merging: on #1257 it was still `pending` when the merge went ahead, so the malformed body of [#1262](https://github.com/mudler/vllm.cpp/issues/1262) landed unread. What is missing is one local call, `gh pr view --json body --jq .body \| check-commit-trailers.py --message-file - --filled`, in the operator's own shell rather than in a queue. NOT fixed in the #1262 flow because it changes an operator procedure and adds a gate command rather than a checker rule, so it owes its own red-first evidence and its own reviewer, and because the lane clears without it. Owned under `## Owed` in [`fix-trailer-lane-cutover.md`](specs/fix-trailer-lane-cutover.md) | bug | +| [#1287](https://github.com/mudler/vllm.cpp/issues/1287) | `ENG-RECORD-ANCHOR-RATCHET` | The record-anchor symbol test asks only whether the cited LINES CONTAIN the symbol, so a COMMENT naming it satisfies the test. `KERNEL-ATTN-MLA-SPARSE` cites `include/vllm/v1/attention/backend.h:271` for `get_kv_cache_shape`, which is a ROCm comment; the declaration is 70 lines down at `:341`, and the anchor passes for the wrong reason. The spec records it as a MEASURED LIMIT rather than a repair, because tightening it would need a parser per language. The second-order cost is the one this issue adds: the ratchet also fails when a bucket FALLS without the baseline being lowered in the same commit, so unrelated drift that parks a comment on a cited line forces the next contributor to bank an improvement that never happened. Worked example on this branch: `72bd06a5a` (a record reconciliation, #535) replaced the `SERVE-ASYNC-LLM` citation `examples/server/main.cpp:230-247` with a bare `examples/server/main.cpp`, so the anchor stopped being counted rather than being repaired, `broken` fell 7 to 6, and the merge had to bank it. NOT #911, which is the different gap that spec BODIES are unpoliced | bug | | [#1246](https://github.com/mudler/vllm.cpp/issues/1246) | `MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation` | MiniMax-Music3: the composed depth SCHEDULE in `Music3DepthStage` (`src/vllm/model_executor/models/minimax_music3_llm.cpp:398-535`) had NO gate — its only two call sites were its own definition and `minimax_music3_llm.cpp:617`, so deleting the 3-row prefix's position-0 K/V append, or silently dropping the fed-back `(index-1)*audio_vocab_size + drawn` projection row (which changes the generated song), each left ALL FIVE music3 suites GREEN. The row's answer was the FNV-1a fingerprint from `tools/bench/music3_depth_stage_ab.cpp`, but that file is a hand TRANSCRIPTION of the schedule rather than a call to `Music3DepthStage`, so it cannot detect divergence between itself and the function it transcribes — and no `CMakeLists.txt` compiled it either. FIXED IN FLOW: `test_minimax_music3_ar` gains a case driving the PRODUCTION function against a transcription of the whole-sequence schedule it replaced, at 8 heads of 8 / 2 layers / 8 codebooks / audio_vocab 32, checkpoint-free, 448 values bitwise plus the drawn codes, the draw count and its own teeth; both mutations now red it and it alone. The driver is compiled by CI as the never-linked OBJECT libraries `vllm_music3_depth_stage_ab_{before,after}`, which caught it failing `-Werror=comment` on the first build. Owned by `MUSIC3-DEPTH-SPEED`, spec §15.5. | bug | | [#1247](https://github.com/mudler/vllm.cpp/issues/1247) | `MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation` | MiniMax-Music3 depth-stage RECORD, four defects, no measured number affected: (a) `test_minimax_music3_ar.cpp`, spec §15.5 and PR #1238's body all say the committed goldens are "8-wide with ONE head" so a head stride is invisible — `minimax_music3_ar_goldens.inc` sets `kMusic3DepthHeads = 2` over `kMusic3DepthHidden = 8`, so they are 2 heads of 4 and dropping the head stride from the cached KEY index reds 4 cases / 32 assertions, TWO of them at the goldens' own geometry; (b) "12 timed rounds per arm" and "all 20 runs printed `f0cfeed6eee4f55d`" cannot be reconciled with the record's own table — 5 pairs x 1 round + 3 pairs x 4 rounds is 17 rounds per arm, and 8 pairs x 2 arms is 16 processes, each printing ONE fingerprint after its round loop; (c) the heading named base `origin/main` `727163997` while the body named `fc163f62b`, and only the latter was built and timed (the delta is #1231's profiler, which the driver never enters); (d) "70 rows a frame to read 16" conflates the 14 rows the OLD arm read with the AFTER arm's 16. FIXED IN FLOW, owned by `MUSIC3-DEPTH-SPEED`, spec §15.2, §15.3, §15.5, §15.6. | bug | | [#1283](https://github.com/mudler/vllm.cpp/issues/1283) | — | **Greedy decode is NOT reproducible at concurrency 16.** Two runs of ONE binary (sha256 `ca114abb…c772ad`) on ONE workload with `--temperature 0 --seed 777 --concurrency 16 --num-prompts 21 --output-len 32`, 23 seconds apart on GB10 / driver `580.173.02` / `VT_ASYNC_RUNNER=0`, emitted DIFFERENT token ids: `5973c5a10a6210085417fb25a29edbd0dc15fe61d7d4f774dd8ff3883dae1d64` (2638 bytes) vs `4cf7923080db6aa29759537f2192f3d9500db11c0e8d72bbb2b4ac6e4614af7c` (2650 bytes). **Not a dedup defect** — `VT_CUDA_GRAPH_DEDUP` is UNSET in both cells; these are the OFF/OFF control of the `ENG-CUDAGRAPH-DEDUP` W5 coarse-key A/B ([#1226](https://github.com/mudler/vllm.cpp/issues/1226)), and the control is the only reason it was seen. Both runs emit `ids_requests=21 ids_total_tokens=672 empty_rows=0`, so the byte delta is JSON decimal width and NOT a length difference, not a truncation and not an early stop; exactly rows 17 and 18 of 21 differ and both diverge MID-DECODE (token index 11 and 5), not at the first token, and both sit in the ragged tail `21 % 16` leaves. Workloads A (conc 24) and C (conc 32) in the same series on the same binary in the same minutes each reproduced themselves exactly, so it is one configuration of three at one repetition each. **The cost: it VOIDS workload B of that run** — `b_off_a == b_exact` and `b_off_a == b_coarse_a` compare against a baseline that does not reproduce itself, and without the OFF/OFF control they would have read as three more byte-identity confirmations. NOT diagnosed beyond the evidence; the issue carries an isolation plan (establish the rate over N repetitions; re-run with `VLLM_CPP_CUDAGRAPH=0` to separate the scheduler from the graph path; a non-ragged `--num-prompts` multiple of `--concurrency`; per-step batch composition for the two diverging requests; the top-2 logit margin at the divergence step, because a near-tie a reduction order can flip is a different defect from a wrong value). Evidence `/mnt/nas_share/rc/dedup-key/logs-ab/ab.log` and `out-ab/ids_b_off_*.json`; owed under `## Owed` of [eng-cudagraph-dedup.md](specs/eng-cudagraph-dedup.md) | bug | diff --git a/.agents/specs/record-anchor-ratchet.md b/.agents/specs/record-anchor-ratchet.md index dcdd7128e..ecd5a52da 100644 --- a/.agents/specs/record-anchor-ratchet.md +++ b/.agents/specs/record-anchor-ratchet.md @@ -1,4 +1,4 @@ -# ENG-RECORD-ANCHOR-RATCHET — the anchor checker sees 18% of its own citations +# ENG-RECORD-ANCHOR-RATCHET — the anchor checker range-checks its own citations and reports nothing Issue: [#632](https://github.com/mudler/vllm.cpp/issues/632) Row: `ENG-RECORD-ANCHOR-RATCHET` ([engine-matrix.md](../engine-matrix.md)) @@ -6,45 +6,81 @@ Row: `ENG-RECORD-ANCHOR-RATCHET` ([engine-matrix.md](../engine-matrix.md)) ## The defect Records cite code as `` `server_main.cpp:505` ``. Code moves; the citation does -not. Nothing catches it — and the checker that appears to is looking at a -different thing. - -`scripts/check-agent-record.py:545`: - -```python -LINK_RE = re.compile(r"\[[^\]]*\]\(([^)]+)\)") -``` - -`local_line_anchors` (`:941`) walks **only** `LINK_RE` matches, and only those -carrying an `#L` fragment (`LINE_FRAGMENT_RE`, `:553`). So the only citation -form it can see is the markdown link `[label](path#L505)`. - -Measured across the five matrices: - -| Matrix | link anchors (seen) | bare `path:line` (invisible) | -|---|---:|---:| -| `engine-matrix.md` | 19 | **1064** | -| `model-matrix.md` | 14 | **662** | -| `kernel-matrix.md` | 144 | 215 | -| `quantization-matrix.md` | 182 | 44 | -| `backend-matrix.md` | 121 | 152 | -| **total** | **480** | **2137** | - -**18.3%** of citations are examined. The dominant form in this repo is the one -the checker never looks at. +not. The checker that appears to catch that reports nothing. + +**This section was wrong when it was written, and the correction is the point of +this row.** It claimed the checker "walks **only** `LINK_RE` matches", so the +bare `` `file.cpp:123` `` form "is never parsed". That is false. +`RAW_LOCAL_ANCHOR_RE` has parsed the bare form since `ee511ca8a` (2026-07-10), +under the prefixes `src`, `include`, `tests`, `examples`, `cmake`, `scripts`, +`tools` and `.github/workflows`, plus `CMakeLists.txt`, and it range-checks each +one. Every one of the 38 offenders this row records sits under those prefixes, +so every one was already parsed. The false premise reached six surfaces before a +fresh review caught it. It is corrected here rather than appended to, so nobody +reads the wrong version first. + +**What the checker actually did.** `local_line_anchors` reads both forms and +range-checks both. What it does not do is REPORT. On a missing file and on an +out-of-range line the loop runs `continue`, so the failing anchor never enters +the returned list. `is_code_anchor` then answers with `any()`, so one good +sibling in the same cell satisfies the row. A bad anchor is therefore invisible +twice: dropped by the parser, then covered by a neighbour. Three gaps compound: -1. **Form.** Bare `` `path:line` `` is not parsed at all (`:941-950`). -2. **`any`, not `all`.** `is_code_anchor` (`:979-985`) returns true if **any** - anchor in the cell qualifies, so one good link covers arbitrarily many rotted - citations beside it. -3. **State.** `EVIDENCED_STATES` (`:530`) omits `ACTIVE` and `READY`, so those - rows get no anchor check at all. +1. **No symbol test.** The check asks whether the line EXISTS, never whether it + holds what the prose says. **32 of the 38** offenders recorded here are IN + RANGE, so a range check could not have found any of them. +2. **`any`, not `all`.** `is_code_anchor` returns true if any anchor in the cell + qualifies, so one good link covers arbitrarily many rotted citations beside + it. That is correct for the STATE gate and the `any` stays; it is why nothing + counted the others. +3. **State.** `EVIDENCED_STATES` omits `ACTIVE` and `READY`, so 92 live rows got + no anchor check at all. + +And a fourth, which is what kept the first three invisible: **there was no +report.** A dropped anchor produced no output at any verbosity. + +### Two populations, two ratios + +Quoting one population's ratio as if it were the other's is what produced the +false claim above, so both are stated with their denominator. Measured at +`8daa67b39`, the head before this branch merged `main` a second time, counting a +citation only where it is the WHOLE of a backtick span, which is what the parser +requires. An earlier count of 2134 used a looser method that +matched a `path:line` token anywhere inside a span, which is why it is larger. + +**Population A — every citation form a reader sees in the five matrices, ours +and upstream:** + +| Matrix | link anchors | bare `path:line` | bare under a `RAW_LOCAL_ANCHOR_RE` prefix | +|---|---:|---:|---:| +| `engine-matrix.md` | 24 | 733 | 366 | +| `model-matrix.md` | 14 | 629 | 82 | +| `quantization-matrix.md` | 182 | 47 | 4 | +| `kernel-matrix.md` | 144 | 155 | 30 | +| `backend-matrix.md` | 128 | 144 | 43 | +| **total** | **492** | **1708** | **525** | + +**1017 of 2200 forms (46.2%) were already parsed.** Most of the rest are +upstream paths (`vllm/model_executor/...py:123`, `csrc/...cu:44`) that no local +checker can validate, which is why this ratio answers no question about coverage +on its own. + +**Population B — the citations this ratchet classifies**, meaning the `code` and +`tests` cells of rows in `RECORD_ANCHOR_STATES` that resolve against this tree. +This is the population that matters: + +| | count | share | +|---|---:|---:| +| in scope | 867 | 100% | +| already parsed AND range-checked before this row | 832 | **96.0%** | +| genuinely new to parsing (`.agents/`, `docs/`, `website/`) | 35 | 4.0% | +| yield no inferable symbol, `OK` by construction | 801 | 92.4% | -And even inside the 18%, `local_line_anchors:957` validates only -`start >= 1 and end <= line_count` — that the line **exists**, never that it is -what the prose says it is. +**Every offender recorded then sat in the 96%.** What this row adds is the symbol test and +the report, not the parser. The parser extension is real but small, and claiming +it as the defect was the error. ## Why a range check is not enough @@ -67,8 +103,8 @@ Every gate stayed green. ## Design — ratchet, not cleanup -Enforcing correctness over 2137 previously-unchecked citations would surface an -unknown backlog in one landing, unrelated to the change itself. Mirror the +Enforcing the symbol test over the whole backlog in one landing would surface an +unknown amount of unrelated rot. Mirror the `device-leakage` shape instead (`scripts/device-leakage-baseline.json`), which this repo already trusts: @@ -89,12 +125,117 @@ ambiguous stays OK. A checker that cries wolf gets disabled. ## Scope -- `ACTIVE`/`READY` join the anchor check as part of this row, since gap 3 is +- `ACTIVE`/`READY` join the anchor COUNT as part of this row, since gap 3 is cheap and the ratchet absorbs whatever it surfaces. +- `EVIDENCED_STATES` itself is deliberately NOT widened, and that is a scope + decision rather than an oversight. Making `ACTIVE` and `READY` *require* an + anchor raises 85 errors across 53 rows that carry prose evidence today, which + is the bulk cleanup this row exists to avoid. (The unit is errors, not rows: + the contract check emits one per missing anchor field, so 32 of the 53 raise + two and 21 raise one.) The 53 are 37 `ACTIVE` and 16 `READY`, and by matrix + 26 engine, 11 backend, 7 model, 6 kernel and 3 quantization. + `RECORD_ANCHOR_STATES` is therefore a separate, wider set. - `is_code_anchor`'s `any` semantics stay for the STATE gate (a row is still evidenced if it has one good anchor); the ratchet counts **every** citation independently, which is where `any` was hiding rot. +## Upstream chain + +**None, and that is a finding rather than an omission.** vLLM has no analogue: +it keeps no stable-ID inventory of its own source, so it has nothing to cite and +nothing to rot. The nearest thing in this tree is the DSR ratchet +(`scripts/check-device-leakage.py`, work row `S1` of +[accelerator-seam-audit.md](accelerator-seam-audit.md)), which is likewise +local-only and is the shape this row mirrors deliberately: same baseline file, +same `--report` / `--write-baseline` idiom, same fails-in-both-directions rule. +The discipline being enforced is AGENTS.md §"vLLM is the reference" — "cite the +`file:line` you ported from" — so this row is what makes that rule checkable +rather than aspirational. + +## Our baseline + +Re-derived at this head after merging 161 commits of `main`, not carried from +any earlier number. The population tables are in `## The defect` above and are +not repeated here. + +**Rot: 38 — 32 `STALE`, 6 `BROKEN`, against 844 `OK`.** + +The set is the 40 the fresh review verified by hand, minus one. +`KERNEL-ATTN-MLA-SPARSE` cites `include/vllm/v1/attention/backend.h:271` for +`get_kv_cache_shape` and now reads `OK`. That is not a repair. The real +declaration is at `:341`; drift on `main` moved a ROCm comment naming the symbol +onto `:271`, and the symbol test asks only whether the cited lines contain the +name. It is a measured limit of the conservative rule, recorded rather than +worked around, because tightening it would need a parser per language. +[#1287](https://github.com/mudler/vllm.cpp/issues/1287) tracks that false +negative, and the second-order cost it carries: because the ratchet fails when a +bucket FALLS without the baseline being lowered in the same commit, unrelated +drift that parks a comment on a cited line forces the next contributor to bank +an improvement that never happened. + +Each merge also turns this row's own anchors `STALE`, by moving the lines they +cite in `check-agent-record.py`. Seven moved on the first merge and five on the +second; both times they were repaired, not banked, because a row arguing that +stale anchors matter may not carry them. **No new offender was banked.** + +The `BROKEN` bucket fell 7 to 6 on the second merge, and `72bd06a5a` is why: +that record reconciliation replaced the `SERVE-ASYNC-LLM` citation +`` `examples/server/main.cpp:230-247` `` with a bare `` `examples/server/main.cpp` ``, +so the anchor stopped being counted rather than being repaired. Banking it was +mandatory under the two-way rule, and it is the worked example behind #1287. + +By top-level directory the 38 are: `src` 24, `include` 5, `examples` 4, +`scripts` 2, `tests` 2, `cmake` 1. Every one is under a prefix the old parser +already read. + +**Specs are not policed by this ratchet.** `RECORD_ANCHOR_FIELDS` reads matrix +row cells, and no spec body is in `MATRIX_PATHS`, so the `file:line` citations +in THIS file are checked by nothing. #911 tracks that gap over 315 spec files. +The anchors here are therefore pinned by hand at the final head and will rot the +same way. Read them as of the commit that lands them. + +## Port map + +Nothing is ported; everything here is written from scratch against the local +record surface, and is recorded as such. + +| Piece | Where | +|---|---| +| bare `` `path:line` `` / `path:line-line` parser | `BARE_CITATION_RE`, `cell_citations` in `scripts/check-agent-record.py` | +| markdown-link parser (retained) | `LINK_RE` + `LINE_FRAGMENT_RE`, read by the same `cell_citations` | +| adjacent-symbol inference | `looks_like_symbol` + the neighbour walk in `cell_citations` | +| OK / STALE / BROKEN classifier | `classify_citation` | +| row and column scope, incl. gap 3 | `RECORD_ANCHOR_STATES`, `RECORD_ANCHOR_FIELDS` | +| the ratchet itself | `check_record_anchors`, `write_record_anchor_baseline`, `scripts/record-anchor-baseline.json` | +| the offender report | `record_anchor_report`, reached by `--report` | + +## Tests to port + +None to port — vLLM has no such checker. The seven cases below are written +RED-first against the shape of the defect, in the file +`scripts/check-pr-size.py` already names as this checker's companion evidence +(`tests/scripts/test_agent_record.py`). + +## Dependencies + +None. No GPU, no build, no network, no new package: the checker is standard +library and the suite runs under `python3 -m unittest`. It depends only on the +row parser already in `scripts/check-agent-record.py`, which is why the ratchet +lives in that file rather than in a new script — a separate checker would have +had to re-derive `parse_claim_rows` and `field_index`, and would have dragged a +`CREATION_MUTATIONS` entry through `scripts/check-pr-size.py` for no gain. + +## Work breakdown + +| Work | Item | +|---|---| +| W1 | The seven RED-first cases, run and captured red before any implementation | +| W2 | Parser: bare citations beside the existing link form | +| W3 | Classifier: OK / STALE / BROKEN, with the conservative symbol rule | +| W4 | Baseline file, the two-way gate, `--report` and `--write-baseline` | +| W5 | Gap 3: `ACTIVE`/`READY` join the counted states | +| W6 | Wiring (`agent-preflight.sh`, the `agent-record` CI job) and the record | + ## Tests RED-first, in `tests/scripts/`: @@ -107,14 +248,32 @@ RED-first, in `tests/scripts/`: | a cell with one good link and one rotted bare citation | the rotted one is still counted (kills `any`) | | an `ACTIVE` row with a rotted anchor | counted (fails today: state excluded) | | baseline raised without a reduction | REFUSED | +| `--write-baseline` on a tree with other record errors | REFUSED, file unchanged (#1270) | The fourth is the load-bearing one — it is the exact shape that let rot hide. Mutate the ratchet into a report-only pass and prove the count case reds. ## Gates -`scripts/agent-preflight.sh --staged`, plus `python3 -m unittest` on the new -suite. No GPU, no build — this is a Python checker and its tests. +No GPU, no build, no network — this is a Python checker and its tests. The +exact invocations, each of which genuinely fails when the row regresses: + +- `python3 scripts/check-agent-record.py --report` — the gate itself; prints + every offender and reds on either direction of the ratchet. +- `python3 tests/scripts/test_agent_record.py` — the mutation suite, including + the ten `RecordAnchorRatchet` cases. +- `scripts/agent-preflight.sh --staged` — the whole record gate over the staged + change. +- `python3 scripts/check-pr-size.py --base origin/main --head HEAD` — proves the + checker change red-before / green-after against its companion suite. +- `python3 scripts/check-commit-trailers.py --range "$(git merge-base origin/main HEAD)..HEAD"` + — run EXPLICITLY over the merge-base range, because `agent-preflight.sh` + silently skips it when the branch is behind `main` (#653). +- `python3 scripts/check-commit-style.py --range "$(git merge-base origin/main HEAD)..HEAD"` + — the same skip applies to the style gate that landed with + `POLICY-SINGLE-PR-AND-STYLE`, so it is run explicitly for the same reason. +- `python3 scripts/check-public-doc-tables.py` — this row writes `docs/STATUS.md` + and `docs/BENCHMARKS.md`, which carry a size ratchet and a per-cell limit. ## Risks / decisions @@ -132,5 +291,54 @@ suite. No GPU, no build — this is a Python checker and its tests. ## Now -`SPIKE` — spec committed, implementation not started. Next: a fresh implementer -takes the parser, the classifier, the baseline and the six cases. +`ACTIVE` — the parser, the classifier, the baseline and the cases are +implemented and green; the gate is wired into `scripts/agent-preflight.sh` (via +`check-agent-record`) and into the `agent-record` CI job as `--report`. + +**Where the conservative line was drawn, and what it cost.** Precision was +chosen over recall at five points, each measured rather than guessed: + +1. **Columns, not paths.** Only `code` and `tests` cells are read. This is what + keeps the upstream citations out of the count structurally; a path-prefix + heuristic would have had to guess, and `tests/`, `cmake/`, `docs/`, `src/` + and `tools/` all collide with upstream references in this record. +2. **Whole-span citations only.** A bare citation must be the entire content of + a backtick span, so prose is never parsed as a path. +3. **Missing files need a parent we own.** A three-component path whose parent + directory exists here is `BROKEN` (`registry.cpp` after the rename to + `model_registry.cpp`); a shallower one is skipped, because llama.cpp's + `src/llama-model.cpp` and vLLM's `cmake/utils.cmake` would otherwise be + blamed on us. +4. **Symbols only from an immediately adjacent backtick span**, with whitespace + or one `(` between. Never from prose. +5. **The span must look like a symbol**: an identifier, 4+ characters, carrying + `_`, `::`, `()` or an uppercase letter, and not starting with `_`. `bf16` and + `nvfp4` sit beside citations constantly and are not symbols. The leading-`_` + exclusion was added after a measured false positive: `` the text-only + `_ModelInfo` `qwen3_5_common.h:42` `` names the `ModelInfo` on that line, and + the anchor was right. + +801 of the 867 in-scope citations yield no inferable symbol and are `OK` by +construction, which is 92.4%, or about 13 in 14. That polarity is the point: a +gate that fires is believed. + +The 38 recorded offenders are deliberately NOT repaired here. They are the +backlog the ratchet exists to hand to whoever next touches each row. + +The first fresh review returned FAIL on the recorded justification and PASS on +the design, so nothing here was redesigned. What changed is the defect +statement, corrected on six surfaces, every ratio restated against its +denominator, and the numbers re-derived after 161 commits of `main`. #1270 was +found and fixed in the same flow: `--write-baseline` banked a baseline from a +tree that had failed other record checks. + +The second fresh review returned FAIL on the record text alone, and the four +findings were record corrections rather than design changes. Two counts were +stale by exactly the case #1270 added, so `## Gates`, the claim and the commit +said ten cases while the row said nine and `## Work breakdown` said six. +`46 rows` was wrong when it was written: `22e6294c6` presented it as re-derived, +and the figure was 50 at that commit and at `8daa67b39`. It is re-derived again +here after the second merge of `main`, which added three `READY` rows and made +it 53. The fourth finding is #1287, filed for the comment-satisfies-symbol false +negative that `## Our baseline` had recorded honestly but without naming its +downstream cost. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4b5acc689..5195da67b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -117,8 +117,13 @@ jobs: with: fetch-depth: 0 - name: Canonical roadmap tables and links are consistent + # --report, wired the way check-device-leakage's is below: the record + # anchor ratchet (ENG-RECORD-ANCHOR-RATCHET, #632) gates on a baseline, + # and printing every STALE/BROKEN offender on every run is what keeps + # the backlog legible instead of a number nobody can act on. It is the + # same gate -- --report adds output, never leniency. run: | - python3 scripts/check-agent-record.py + python3 scripts/check-agent-record.py --report python3 tests/scripts/test_agent_record.py python3 tests/scripts/test_check_issue_index_append_only.py python3 tests/scripts/test_doc_checkpoint.py diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 7f965fc7e..2a2ae8a24 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -11,6 +11,7 @@ | **Container images (ACTIVE; arm64 cuda verified on GB10 + Orin 2026-08-11)** | `ENG-RELEASE-CONTAINERS` ([spec](../.agents/specs/container-images.md)) | cpu amd64 783 MB; cuda arm64 **1.71 GB**. GB10 `sm_121a`: `/health`+`/version`+SIGTERM on `--gpus all`. Orin `sm_87` (Tegra): Qwen3-0.6B **generates**, GPU **GR3D 95-97%** | n/a | | **Developer/row protocol** | Contribution entry point; `ENG-NOW-DERIVED` #374 @`dbd0d51c` | Entry-point gates retained. #374 W1-W5 DONE; benchmark/runtime/parity `VOID`; row specs now carry `## Now` | n/a | | **NemotronH paged forward** (`MODEL-NEMOTRON-H-ABI-A2P`, [#810](https://github.com/mudler/vllm.cpp/issues/810)) | **No speed number, by the unit's own rule** ([spec](../.agents/specs/nemotron-h-a2p-paged-forward.md) §5) | **A3 host gate PASSES 96/96 `STRICT PASS`.** GB10 read 4/24; cause and fix [#1157](https://github.com/mudler/vllm.cpp/issues/1157), sm_121a re-run pending a lease | CPU gate 12/12. Load 209.0 s, peak RSS 20 142 392 KB | +| **Record-anchor ratchet** (`ENG-RECORD-ANCHOR-RATCHET`, #632) | **No number owed:** a record checker. At `8daa67b39`, **832 of 867** in-scope citations (**96.0%**) were already parsed; no symbol test and no report ran. Rot **38** (32 stale, 6 broken) over **844** OK, **32 in range** | | **LoRA runtime W2** (`LORA-RUNTIME`, #278) | **No number owed:** correctness-only; a grid PENDS the W7 model gate | | **ARCH audit: ABI is text-only** | 4 capabilities (H3 video, Laguna, Kimi-Linear, DeepSeek-V4) reachable only from `examples/`, none registry-backed. No gate asks whether a CONSUMER can reach a capability. Documentation only | | **CUDA-graph break seam W1** (`ENG-CUDAGRAPH-BREAK`, [#1192](https://github.com/mudler/vllm.cpp/issues/1192)) | **No number owed, and none taken:** coverage and correctness row, no throughput gate declared | Capability, not a rate: mid-forward capture re-begin holds on a leased GPU; scoped forward matches eager, 500 logits, 0 differing. Probe committed, recipe and sha256 in the [record](../.agents/benchmark-record.md) | diff --git a/docs/STATUS.md b/docs/STATUS.md index c5c461c39..5ece133c7 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -91,6 +91,15 @@ Lifecycle changes still update STATUS and BENCHMARKS, while the moved row spec's `## Now` replaces the per-row write to `.agents/NOW.md`. Runtime and performance are `VOID`; no product behavior changed. +Record-anchor ratchet (2026-08-18, `ENG-RECORD-ANCHOR-RATCHET` `ACTIVE`, #632): +the record's own `file.cpp:123` citations were range-checked and never reported. +Both forms were already parsed; a failing anchor was dropped and covered by +`any()`, and no symbol test ran. 32 of the 38 offenders are in range. +`check-agent-record.py` now classifies each citation OK, STALE or BROKEN and +gates the rot against `scripts/record-anchor-baseline.json` in both directions. +Backlog **38** (32 stale, 6 broken) over 844 correct; `ACTIVE` and `READY` rows +join the count. No product behavior changed. + Supported-model registry guard (2026-08-06): the public per-architecture list in [FEATURES](FEATURES.md) is CI-bound to the C++ registry by `scripts/check-supported-models.py` (+ mutation test), so the 37 diff --git a/scripts/agent-preflight.sh b/scripts/agent-preflight.sh index 971da0950..a1b4c5daf 100755 --- a/scripts/agent-preflight.sh +++ b/scripts/agent-preflight.sh @@ -318,6 +318,14 @@ for checker in "${CHECKERS[@]}"; do # is the flag. Wiring either without --check installs a gate that cannot # fail, which for check-gate-commands is the very defect it classifies. claim-view|check-gate-commands) run "$checker" python3 "scripts/$checker.py" --check ;; + # check-agent-record also carries the record-anchor ratchet + # (ENG-RECORD-ANCHOR-RATCHET, #632): a citation that names a line no longer + # holding the symbol beside it fails HERE, on the plain call, and the error + # names the bucket that moved. Deliberately NOT wired as --report: `run` + # shows only the first 12 lines of a failure, and the report's offender list + # would push the error message out of that window. The full list is one + # command away (`scripts/check-agent-record.py --report`) and CI prints it + # unconditionally. *) run "$checker" python3 "scripts/$checker.py" ;; esac done diff --git a/scripts/check-agent-record.py b/scripts/check-agent-record.py index 0697b5371..37baa0480 100644 --- a/scripts/check-agent-record.py +++ b/scripts/check-agent-record.py @@ -3,6 +3,9 @@ from __future__ import annotations +import argparse +import dataclasses +import json import re import subprocess import sys @@ -528,13 +531,20 @@ # is absent at the pin and the gate runs against llama.cpp `237ad9b96`. # `READY`, spec `specs/hybrid-placement.md`, issue #149. # 156 since 2026-08-14: +`ENG-RECORD-ANCHOR-RATCHET` (the record's own `path:line` -# citations are 82% unchecked -- `LINK_RE` in THIS file matches only markdown -# links, so 2137 bare `file.cpp:123` citations across the five matrices are never -# parsed, against 480 that are; and the 18% seen is only range-checked, never -# checked to contain the symbol named beside it). Found by three stale anchors that -# humans caught by reading during the 2026-08-13/14 campaign, all of them IN RANGE. -# Issue #632; `SPIKE` on its committed spec. The row claims no implementation: no -# parser, no baseline and no test exists yet. +# citations were checked for RANGE but never for CONTENT, and a failing check was +# silently DROPPED. `local_line_anchors` in THIS file parses both citation forms +# -- markdown links, and (since ee511ca8a) bare `file.cpp:123` under the +# `RAW_LOCAL_ANCHOR_RE` prefixes -- but on a missing file or an out-of-range line +# it `continue`s, so the bad anchor is omitted from the list and swallowed by +# `is_code_anchor`'s `any()`. There was no symbol test and no report. 32 of the +# 38 offenders this row banks are IN RANGE, so range-checking alone could never +# have found them. Found by three stale anchors that humans caught by reading +# during the 2026-08-13/14 campaign, all of them IN RANGE. Issue #632. +# It lands `ACTIVE`, not `SPIKE`: the same change carries the parser, the +# STALE/BROKEN classifier, `scripts/record-anchor-baseline.json` and the +# `RecordAnchorRatchet` suite, so a comment framed at the spec-only commit would +# be false about the file it sits in. The COUNT is unchanged by that -- the row +# already existed at 156 and this is not a bump. # 157 since 2026-08-17: +`ENG-RESIDENCY-CONFIG` (the host-RAM->DISK weight-residency # tier as a CONFIG surface -- a `vllm_cpp` extension key inside the existing # `--offload-config` document, reaching the loader through @@ -1086,6 +1096,370 @@ def ledger_line_anchors(value: str, source: Path) -> list[str]: ] +# --- record-anchor ratchet (ENG-RECORD-ANCHOR-RATCHET, #632) ------------------ +# +# Records cite code as `server_main.cpp:505`. Code moves; the citation does not. +# The checker above LOOKS like it catches that and does not: +# +# 1. NO REPORT, not "no parser". `local_line_anchors` reads BOTH citation +# forms: the markdown link through `LINK_RE`, and the bare +# `file.cpp:123` form through `RAW_LOCAL_ANCHOR_RE` since ee511ca8a. It +# range-checks each one. What it does not do is REPORT: on a missing file +# and on an out-of-range line the loop runs `continue`, so the bad anchor +# never enters the returned list, and `is_code_anchor`'s `any()` swallows +# what is left. There was no symbol test either, which is the gap that +# matters: 32 of the 38 offenders recorded here are IN RANGE, so a range +# check could not have found them. +# 2. `any`, NOT `all`. `is_code_anchor` returns true if ONE anchor in a cell +# qualifies, so a single good link covers arbitrarily many rotted citations +# beside it. That is not a bug in the STATE gate -- a row IS evidenced by +# one good anchor, and the `any` stays -- but it is why nothing counted the +# others. +# 3. STATE. `EVIDENCED_STATES` omits `ACTIVE` and `READY`, so 92 live rows got +# no anchor check at all. +# +# TWO POPULATIONS, TWO RATIOS, and quoting one without its denominator is what +# produced the false "the bare form was never parsed" claim this comment +# replaces. Measured at `8daa67b39`, the head before this branch merged `main` a +# second time, counting a citation only where it is the WHOLE of a backtick +# span, which is what the parser requires: +# +# * ALL citation forms in the five matrices, ours and upstream: 492 links and +# 1708 bare, 2200 in total. 525 of the bare forms sit under a +# `RAW_LOCAL_ANCHOR_RE` prefix, so 1017 of 2200 (46.2%) were already parsed. +# Most of the remainder are upstream paths that reach no local checker. +# * The citations this ratchet CLASSIFIES, which is the population that +# matters: 867. Of those 832 (96.0%) were already parsed AND range-checked +# before this row, and 35 (4.0%) are genuinely new to parsing, under +# `.agents/`, `docs/` and `website/`. Every offender recorded then sat in +# the 96%. The value this row adds is the symbol test and the report, not the +# parser. +# +# A range check is not the fix: every stale anchor found +# by hand during the 2026-08-13/14 campaign was IN RANGE -- +# `docs/USAGE.md:902` (the count line had moved to :1126), `multimodal.py:17-43` +# (the block ends at :45) and `server_main.cpp:308` (a different table entry +# after a 4-line comment landed above). Only "does this line contain the symbol +# named beside it" separates those from a live citation. +# +# THE RATCHET. Enforcing correctness over the whole backlog in one landing would +# surface an unknown amount of unrelated rot, so this mirrors the DSR ratchet in +# scripts/check-device-leakage.py, which this repo already trusts: +# scripts/record-anchor-baseline.json holds the accepted STALE + BROKEN counts, +# a bucket ABOVE its baseline fails, and a bucket BELOW it fails too, with the +# instruction to lower the baseline in the SAME commit as the repair. The number +# only ever moves down, and only deliberately. `--report` names every offender +# so the backlog is legible rather than a number. +# +# WHERE THE CONSERVATIVE LINE IS DRAWN, and why each side of it is where it is. +# A checker that cries wolf gets disabled, and this one has to survive a +# four-figure backlog, so every rule below prefers a missed rot to a false one: +# +# * ONLY the `code` and `tests` cells of rows in RECORD_ANCHOR_STATES. The +# `upstream` column is never read. That is what keeps the upstream +# references (`vllm/model_executor/...py:123`, `csrc/...cu:44`) out of the +# count structurally, rather than by a path heuristic. +# * A bare citation must be the WHOLE of a backtick span, so running prose can +# never be parsed as a path. +# * It must resolve to a file in the tree, or be a near miss: at least two path +# separators AND an existing parent directory, i.e. "a directory we own with +# a filename we do not" -- a rename or a deletion. `vllm/utils/hashing.py` +# and `tests/v1/core/test_scheduler.py` have no such parent here and are +# silently skipped, which is correct: they are upstream, and we cannot +# validate an anchor into a tree we do not have. +# * A symbol is inferred ONLY from an immediately adjacent backtick span -- +# whitespace between them, or whitespace and one `(` for the trailing +# `path:line (`Symbol`)` form. Nothing else, and never from prose. +# * That span must LOOK like a symbol: an identifier, at least 4 characters, +# carrying `_`, `::`, `()` or an uppercase letter. `bf16` and `nvfp4` sit +# next to citations constantly and are not symbols; `MoeAuxStream`, +# `evict_blocks` and `Scheduler::shutdown()` are. +# * With no inferable symbol the citation is OK by construction. 801 of the +# 867 in-scope citations land there, which is 92.4%, or about 13 in 14. +# That is the intended polarity: this gate exists to be believed when it +# does fire. +# * The symbol test asks whether the cited LINES CONTAIN the name. A comment +# that mentions the symbol therefore reads OK. That is a measured limit and +# not a defect: `KERNEL-ATTN-MLA-SPARSE` cites +# `include/vllm/v1/attention/backend.h:271` for `get_kv_cache_shape`, whose +# real declaration is at :341, and drift on `main` moved a ROCm comment +# naming the symbol onto :271. Tightening this would need a parser per +# language, which is the cry-wolf trade this whole block refuses. +RECORD_ANCHOR_BASELINE = ROOT / "scripts/record-anchor-baseline.json" +RECORD_ANCHOR_VERDICTS = ("ok", "stale", "broken") +# The BUDGET is the rot only. `ok` is counted and printed but deliberately NOT +# stored: a baseline that pinned it would make every PR that adds or removes any +# citation rewrite this file, which is exactly the shared-file lock AGENTS.md +# forbids. Per-bucket rather than one total, so a repaired BROKEN cannot pay for +# a new STALE. +RECORD_ANCHOR_BUCKETS = ("stale", "broken") +# Gap 3. `EVIDENCED_STATES` itself is deliberately NOT widened. Making ACTIVE and +# READY *require* an anchor raises 85 errors across 53 rows that carry prose +# evidence today, which is the bulk cleanup this row exists to avoid. (The unit +# is errors, not rows: the contract check emits one per missing anchor field, so +# a row can raise more than one -- 32 rows raise two here and 21 raise one.) +# They join the COUNT instead, and the ratchet absorbs what that surfaces. +RECORD_ANCHOR_STATES = EVIDENCED_STATES | {"ACTIVE", "READY"} +RECORD_ANCHOR_FIELDS = ("code", "tests") +BARE_CITATION_RE = re.compile( + r"([A-Za-z0-9_./+-]*[A-Za-z0-9_+-]\.[A-Za-z0-9_+-]+):(\d+)(?:-(\d+))?" +) +BACKTICK_SPAN_RE = re.compile(r"`([^`\n]+)`") +SYMBOL_RE = re.compile(r"[A-Za-z_][A-Za-z0-9_]*(?:::[A-Za-z_][A-Za-z0-9_]*)*(?:\(\))?") + + +@dataclass(frozen=True) +class Citation: + # No matrix field: row IDs are unique across the matrices (check_matrices + # enforces it), so the item id already locates the offender. + item_id: str + path: str + start: int + end: int + symbol: str | None + verdict: str + + def describe(self) -> str: + span = f"{self.start}" if self.end == self.start else f"{self.start}-{self.end}" + want = f" expected `{self.symbol}`" if self.symbol else "" + return f"{self.verdict:<6} {self.item_id} -> {self.path}:{span}{want}" + + +@dataclass +class RecordAnchorResult: + counts: dict[str, int] = dataclasses.field( + default_factory=lambda: dict.fromkeys(RECORD_ANCHOR_VERDICTS, 0) + ) + citations: list[Citation] = dataclasses.field(default_factory=list) + + @property + def offenders(self) -> list[Citation]: + return [c for c in self.citations if c.verdict in {"STALE", "BROKEN"}] + + @property + def total(self) -> int: + """STALE + BROKEN. `ok` is reported but is not part of the budget.""" + return sum(self.counts[bucket] for bucket in RECORD_ANCHOR_BUCKETS) + + +def looks_like_symbol(text: str) -> bool: + text = text.strip() + if len(text) < 4 or SYMBOL_RE.fullmatch(text) is None: + return False + # A LEADING underscore in a record is nearly always an abbreviated suffix -- + # "the text-only `_ModelInfo`" naming the `ModelInfo` beside it, not a + # symbol spelled `_ModelInfo`. Searching for it literally produced the one + # false STALE this rule was measured against, on an anchor that was right. + if text.startswith("_"): + return False + return "_" in text or "::" in text or text.endswith("()") or any(c.isupper() for c in text) + + +def cell_citations(cell: str, source: Path, root: Path) -> list[tuple[Path, int, int, str | None]]: + """Every citation of THIS tree in one record cell, with its expected symbol. + + Returns `(resolved_path, start, end, symbol_or_None)`. Both citation forms + are recognised -- the markdown link `[label](path#L12)` the old parser saw, + and the bare `` `path:12` `` / `` `path:12-20` `` span that is six times + more common here and was never parsed as a citation at all. + """ + tokens: list[tuple[int, int, str, object]] = [] + links: list[tuple[int, int]] = [] + for match in LINK_RE.finditer(cell): + target = match.group(1).strip().strip("<>") + links.append((match.start(), match.end())) + if not target or target.startswith(("http://", "https://", "mailto:")): + continue + target_path, _, fragment = target.partition("#") + line_match = LINE_FRAGMENT_RE.fullmatch(fragment) + if line_match is None: + continue + start = int(line_match.group(1)) + end = int(line_match.group(2) or start) + resolved = (source.parent / target_path).resolve() + tokens.append((match.start(), match.end(), "citation", (resolved, start, end))) + for match in BACKTICK_SPAN_RE.finditer(cell): + # A backtick span inside a link is that link's LABEL, not a token beside + # it: `[`foo.cpp`](../src/foo.cpp#L4)` must not be read as a neighbour. + if any(lo <= match.start() < hi for lo, hi in links): + continue + inner = match.group(1).strip() + bare = BARE_CITATION_RE.fullmatch(inner) + if bare is not None: + start = int(bare.group(2)) + end = int(bare.group(3) or start) + resolved = (root / bare.group(1)).resolve() + tokens.append((match.start(), match.end(), "citation", (resolved, start, end))) + elif looks_like_symbol(inner): + tokens.append((match.start(), match.end(), "symbol", inner)) + else: + tokens.append((match.start(), match.end(), "other", inner)) + tokens.sort(key=lambda t: t[0]) + + found: list[tuple[Path, int, int, str | None]] = [] + for index, (lo, hi, kind, payload) in enumerate(tokens): + if kind != "citation": + continue + resolved, start, end = payload # type: ignore[misc] + symbol: str | None = None + if index > 0 and tokens[index - 1][2] == "symbol": + if not cell[tokens[index - 1][1]:lo].strip(): + symbol = str(tokens[index - 1][3]).strip() + if symbol is None and index + 1 < len(tokens) and tokens[index + 1][2] == "symbol": + if cell[hi:tokens[index + 1][0]].strip() in {"", "("}: + symbol = str(tokens[index + 1][3]).strip() + found.append((resolved, start, end, symbol)) + return found + + +def classify_citation( + resolved: Path, start: int, end: int, symbol: str | None, root: Path +) -> str | None: + """OK / STALE / BROKEN, or None when the citation is not about this tree.""" + if not resolved.is_file(): + # "A directory we own with a filename we do not" -- a rename or a + # deletion, and the one missing-file case worth calling BROKEN. Require + # THREE path components so a one-segment upstream name whose top-level + # directory happens to match ours (llama.cpp's `src/llama-model.cpp`, + # vLLM's `cmake/utils.cmake`) is skipped rather than blamed on us. + try: + depth = len(resolved.relative_to(root).parts) + except ValueError: + return None + if depth >= 3 and resolved.parent.is_dir(): + return "BROKEN" + return None + lines = resolved.read_text(encoding="utf-8", errors="replace").splitlines() + if start < 1 or end < start or end > len(lines): + return "BROKEN" + if symbol is None: + return "OK" + base = symbol.removesuffix("()").split("::")[-1] + body = "\n".join(lines[start - 1:end]) + return "OK" if re.search(r"\b" + re.escape(base) + r"\b", body) else "STALE" + + +def scan_record_anchors( + rows: list[ClaimRow] | None = None, root: Path | None = None +) -> RecordAnchorResult: + root = ROOT if root is None else root + if rows is None: + rows, _ = check_matrices([]) + result = RecordAnchorResult() + for row in rows: + if row.state not in RECORD_ANCHOR_STATES: + continue + # BY INDEX, not by name: several matrices carry one `Local evidence` + # column that field_index resolves for BOTH `code` and `tests`, and + # reading it twice would count every citation in it twice. + indices = {field_index(row.header, name) for name in RECORD_ANCHOR_FIELDS} + for index in sorted(i for i in indices if i is not None): + cell = row.cells[index] if index < len(row.cells) else "" + if is_placeholder(cell): + continue + for resolved, start, end, symbol in cell_citations(cell, row.path, root): + verdict = classify_citation(resolved, start, end, symbol, root) + if verdict is None: + continue + try: + shown = resolved.relative_to(root).as_posix() + except ValueError: + shown = resolved.as_posix() + result.counts[verdict.lower()] += 1 + result.citations.append( + Citation( + item_id=row.item_id, + path=shown, + start=start, + end=end, + symbol=symbol, + verdict=verdict, + ) + ) + return result + + +def load_record_anchor_baseline() -> dict[str, int]: + if not RECORD_ANCHOR_BASELINE.is_file(): + return {} + data = json.loads(RECORD_ANCHOR_BASELINE.read_text(encoding="utf-8")) + return {bucket: int(data["buckets"][bucket]) for bucket in RECORD_ANCHOR_BUCKETS} + + +def write_record_anchor_baseline(result: RecordAnchorResult) -> int: + previous = load_record_anchor_baseline() + if previous and result.total > sum(previous.values()): + print( + "REFUSING to write a HIGHER record-anchor baseline " + f"({sum(previous.values())} -> {result.total}). The ratchet only turns one " + "way: repair the anchors instead of banking the rot.", + file=sys.stderr, + ) + return 1 + payload = { + "_comment": [ + "Record-anchor baseline for scripts/check-agent-record.py", + "(ENG-RECORD-ANCHOR-RATCHET, .agents/specs/record-anchor-ratchet.md).", + "STALE = the cited line exists but does not contain the symbol named", + "beside it. BROKEN = the line is out of range, or the file is gone.", + "THESE NUMBERS MAY ONLY EVER GO DOWN. Lower them in the SAME commit as", + "the repair that earned it, by running:", + " python3 scripts/check-agent-record.py --write-baseline", + "It is a rot budget, never to be raised to make a failing check pass.", + "Only the rot is stored. The OK count is printed by --report but kept out", + "of this file on purpose: pinning it would make every change that adds or", + "removes a citation rewrite this file, which is a lock, not a ratchet.", + ], + "total": result.total, + "buckets": {bucket: result.counts[bucket] for bucket in RECORD_ANCHOR_BUCKETS}, + } + RECORD_ANCHOR_BASELINE.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8") + print(f"baseline written: {RECORD_ANCHOR_BASELINE.name} -> {result.total}") + return 0 + + +def record_anchor_report(result: RecordAnchorResult) -> str: + lines = [ + "Record anchors in the `code` / `tests` cells of " + f"{'/'.join(sorted(RECORD_ANCHOR_STATES))} rows:", + "", + ] + lines.extend(f" {c.describe()}" for c in result.offenders) + if result.offenders: + lines.append("") + lines.append( + "record anchors: " + + ", ".join(f"{b}={result.counts[b]}" for b in RECORD_ANCHOR_VERDICTS) + + f" -> rot {result.total}" + ) + return "\n".join(lines) + + +def check_record_anchors(result: RecordAnchorResult, errors: list[str]) -> None: + baseline = load_record_anchor_baseline() + if not baseline: + errors.append( + f"no record-anchor baseline at {RECORD_ANCHOR_BASELINE.relative_to(ROOT)}; " + "run --write-baseline to establish one" + ) + return + for bucket in RECORD_ANCHOR_BUCKETS: + got, want = result.counts[bucket], baseline[bucket] + if got > want: + errors.append( + f"RECORD ANCHOR REGRESSION in bucket '{bucket}': {got} > baseline {want}. " + "A citation names a line that no longer holds what the prose says it " + "does. Run `python3 scripts/check-agent-record.py --report` for the " + "offenders and repair the anchor. NEVER raise the baseline to pass." + ) + elif got < want: + errors.append( + f"record-anchor baseline STALE in bucket '{bucket}': {got} < baseline " + f"{want}. A repair must lower the baseline in the SAME commit: run " + "`python3 scripts/check-agent-record.py --write-baseline` and commit it." + ) + + def commit_exists(commit: str) -> bool: result = subprocess.run( ["git", "cat-file", "-e", f"{commit}^{{commit}}"], @@ -1566,7 +1940,20 @@ def check_roadmap(by_id: dict[str, ClaimRow], errors: list[str]) -> None: errors.append(f"{source.relative_to(ROOT)}: references unknown stable row {item_id}") -def main() -> int: +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--report", + action="store_true", + help="print every STALE/BROKEN record anchor with the symbol expected", + ) + parser.add_argument( + "--write-baseline", + action="store_true", + help="rewrite scripts/record-anchor-baseline.json (only ever DOWNWARD)", + ) + args = parser.parse_args(argv) + errors: list[str] = [] for path in REQUIRED: if not path.is_file(): @@ -1602,11 +1989,33 @@ def main() -> int: check_spec_location(errors) check_roadmap(by_id, errors) + anchors = scan_record_anchors(rows) + if args.report: + print(record_anchor_report(anchors)) + if not args.write_baseline: + # Writing is a MODE, not a step: it must not also report the gate it + # is about to move, or a run that lowered the baseline would print a + # regression against the value it just replaced. + check_record_anchors(anchors, errors) + elif args.write_baseline: + print( + "REFUSING to write a baseline from a tree whose record does not parse.", + file=sys.stderr, + ) + if errors: for error in dict.fromkeys(errors): print(f"ERROR: {error}", file=sys.stderr) return 1 + if args.write_baseline: + # AFTER the error gate, deliberately. The mode used to return the moment + # it had a number, so a tree that failed some OTHER record check could + # still bank its rot, and the banked figure would carry the authority of + # a run that never passed. A baseline is a measurement of the record, so + # it is only taken from a record that checks out. + return write_record_anchor_baseline(anchors) + counts = [ "ENGINE=" + str( @@ -1622,6 +2031,7 @@ def main() -> int: f"{prefix}=" + str(sum(row.item_id.startswith(prefix + "-") for row in rows if row.path == path)) ) + counts.append(f"ANCHOR-ROT={anchors.total}") print("agent record OK: " + " ".join(counts)) return 0 diff --git a/scripts/check-gate-commands.py b/scripts/check-gate-commands.py index aaa48bc9a..cee129505 100755 --- a/scripts/check-gate-commands.py +++ b/scripts/check-gate-commands.py @@ -346,6 +346,15 @@ def audit() -> list[dict]: # focused test file, and records that no CUDA/GPU/SACRED/oracle gate is # implicated because the change is argument parsing and reaches no forward pass. # Growth from a lifecycle move, so the set is re-pinned in the same change. +# 2026-08-14: +ENG-RECORD-ANCHOR-RATCHET. The row leaves SPIKE for ACTIVE on its +# implementation (issue #632), which puts it in GATED_STATES for the first time. +# Its spec's Gates section names five invocations, and this is a STRONG credit +# rather than one of the weak ones described above: the row's gate IS +# `scripts/check-agent-record.py`, so the credited command is the thing under +# test, and it fails on either direction of the ratchet. The suite +# (`tests/scripts/test_agent_record.py`) is proven red against the BASE checker +# by `scripts/check-pr-size.py`, which is itself one of the five. Growth from a +# lifecycle move, so the set is re-pinned in the same change. # 2026-08-16: +SPEC-MTP-K-GT-1. A NEW row arriving at ACTIVE (issue #81), so it # enters GATED_STATES for the first time. Its spec's Gates section names # `scripts/agent-preflight.sh` plus the built CPU suite (493 passed / 0 failed / @@ -380,6 +389,7 @@ def audit() -> list[dict]: "ENG-CUDAGRAPH-DEDUP", "SPEC-MTP-K-GT-1", "ATTN-CHUNKED-LOCAL", + "ENG-RECORD-ANCHOR-RATCHET", "SERVE-RECIPE-ARGS", "ENG-FORGE-COAUTHOR", "ENG-RECORD-CONFLICT-SURFACES", diff --git a/scripts/record-anchor-baseline.json b/scripts/record-anchor-baseline.json new file mode 100644 index 000000000..89a63633b --- /dev/null +++ b/scripts/record-anchor-baseline.json @@ -0,0 +1,20 @@ +{ + "_comment": [ + "Record-anchor baseline for scripts/check-agent-record.py", + "(ENG-RECORD-ANCHOR-RATCHET, .agents/specs/record-anchor-ratchet.md).", + "STALE = the cited line exists but does not contain the symbol named", + "beside it. BROKEN = the line is out of range, or the file is gone.", + "THESE NUMBERS MAY ONLY EVER GO DOWN. Lower them in the SAME commit as", + "the repair that earned it, by running:", + " python3 scripts/check-agent-record.py --write-baseline", + "It is a rot budget, never to be raised to make a failing check pass.", + "Only the rot is stored. The OK count is printed by --report but kept out", + "of this file on purpose: pinning it would make every change that adds or", + "removes a citation rewrite this file, which is a lock, not a ratchet." + ], + "total": 38, + "buckets": { + "stale": 32, + "broken": 6 + } +} diff --git a/tests/scripts/test_agent_record.py b/tests/scripts/test_agent_record.py index c98a913a8..e1596effd 100644 --- a/tests/scripts/test_agent_record.py +++ b/tests/scripts/test_agent_record.py @@ -3,8 +3,10 @@ from __future__ import annotations +import contextlib import importlib.util import io +import json import re import sys import tempfile @@ -297,11 +299,11 @@ def test_anchor_ratchet_row_is_inside_the_engine_ratchet(self) -> None: Same shape as the #117, #606 and #633 assertions above. Worth naming here for one reason beyond the count: this row exists BECAUSE the - `path:line` citations in these matrices are 82% unparsed by the very - checker this test guards, so the row's own anchors into - `check-agent-record.py` are — until it lands — as unchecked as the ones - it is filed about. Pinning the row is the only mechanical statement - available about it today. + `path:line` citations in these matrices were 83% unparsed by the very + checker this test guards. That is no longer true, and the row is now + the first thing its own ratchet polices -- `RecordAnchorRatchet` below + counts the anchors in this row's `Our code` cell like any other. Pinning + the row still says the thing a count cannot: that it exists. """ errors: list[str] = [] @@ -1392,6 +1394,177 @@ def test_owed_issues_reads_specs_with_a_glob(self) -> None: self.assertIsInstance(agent_record.owed_issues(), set) +class RecordAnchorRatchet(unittest.TestCase): + """ENG-RECORD-ANCHOR-RATCHET (#632), .agents/specs/record-anchor-ratchet.md. + + Ten cases. Seven cover a row of the spec's test table; the other three pin + the two gate directions and the tree-against-baseline agreement. The + table-driven cases build a SYNTHETIC tree and a synthetic row rather than + asserting against the live matrices, because the live count is a moving + backlog and a case that reds when somebody else repairs an unrelated anchor + teaches people to ignore this suite. + + The fourth case is the load-bearing one. `is_code_anchor` answers with + `any()`, so before this row one good link in a cell made every rotted + citation beside it invisible -- and that is the exact shape three stale + anchors hid in during the 2026-08-13/14 campaign. + """ + + HEADER = ("id", "item", "upstream", "our code", "tests evidence", "state", "owner") + + def row(self, state: str, code: str, tests: str = "-", *, source=None): + cells = ("ENG-RATCHET-FIXTURE", "item", "up", code, tests, f"`{state}`", "-") + return agent_record.ClaimRow( + path=source if source is not None else agent_record.ENGINE_MATRIX, + line_no=1, + item_id="ENG-RATCHET-FIXTURE", + state=state, + header=self.HEADER, + cells=cells, + raw="| " + " | ".join(cells) + " |", + ) + + @staticmethod + def tree(root: Path) -> None: + """A cited file whose symbol sits at :4, not at :2.""" + target = root / "src/vllm/toy.cpp" + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text( + "#include \n" # 1 + "// a comment that moved\n" # 2 + "\n" # 3 + "void RatchetTarget() {}\n", # 4 + encoding="utf-8", + ) + + def scan(self, root: Path, rows): + return agent_record.scan_record_anchors(rows, root=Path(root)) + + def test_bare_citation_at_the_wrong_line_counts_stale(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + self.tree(Path(tmp)) + res = self.scan(tmp, [self.row("PARTIAL", "`RatchetTarget` `src/vllm/toy.cpp:2`")]) + self.assertEqual(res.counts["stale"], 1, res.offenders) + self.assertEqual(res.counts["broken"], 0, res.offenders) + self.assertEqual(res.counts["ok"], 0, res.offenders) + + def test_bare_citation_out_of_range_counts_broken(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + self.tree(Path(tmp)) + res = self.scan(tmp, [self.row("PARTIAL", "`RatchetTarget` `src/vllm/toy.cpp:99`")]) + self.assertEqual(res.counts["broken"], 1, res.offenders) + self.assertEqual(res.counts["stale"], 0, res.offenders) + + def test_correct_bare_citation_counts_ok(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + self.tree(Path(tmp)) + res = self.scan(tmp, [self.row("PARTIAL", "`RatchetTarget` `src/vllm/toy.cpp:4`")]) + self.assertEqual(res.counts["ok"], 1, res.offenders) + self.assertEqual(res.total, 0, res.offenders) + + def test_one_good_link_does_not_cover_a_rotted_bare_citation(self) -> None: + """The `any()` shape that hid the rot: BOTH citations must be counted. + + Built against REAL tree paths rather than a synthetic root, because the + half of the claim that matters is the interaction with `is_code_anchor`, + and that function resolves against `ROOT` by construction. `:1` of this + checker is its shebang -- in range, and forever without the symbol the + cell names beside it. + """ + cell = ( + "[checker](../scripts/check-agent-record.py#L1); " + "`RatchetFixtureSymbol` `scripts/check-agent-record.py:1`" + ) + source = agent_record.ENGINE_MATRIX + # `any()` semantics are DELIBERATELY retained for the STATE gate -- one + # good anchor still evidences the row (spec, "Scope"). Before this row + # that was ALSO the whole of the anchor check, so the rotted citation + # beside it was invisible. + self.assertTrue(agent_record.is_code_anchor(cell, source)) + res = agent_record.scan_record_anchors([self.row("PARTIAL", cell, source=source)]) + self.assertEqual(res.counts["ok"], 1, res.offenders) + self.assertEqual(res.counts["stale"], 1, res.offenders) + + def test_active_row_anchors_are_counted(self) -> None: + """EVIDENCED_STATES omits ACTIVE, so this row got no anchor check at all.""" + with tempfile.TemporaryDirectory() as tmp: + self.tree(Path(tmp)) + res = self.scan(tmp, [self.row("ACTIVE", "`RatchetTarget` `src/vllm/toy.cpp:2`")]) + self.assertEqual(res.counts["stale"], 1, res.offenders) + self.assertIn("ACTIVE", agent_record.RECORD_ANCHOR_STATES) + self.assertIn("READY", agent_record.RECORD_ANCHOR_STATES) + + def test_write_baseline_refuses_to_ratchet_upward(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + baseline = Path(tmp) / "record-anchor-baseline.json" + baseline.write_text( + json.dumps({"total": 1, "buckets": {"stale": 1, "broken": 0}}) + "\n", + encoding="utf-8", + ) + before = baseline.read_text(encoding="utf-8") + result = agent_record.RecordAnchorResult() + result.counts["stale"] = 3 + result.counts["broken"] = 1 + err = io.StringIO() + with contextlib.redirect_stderr(err), mock.patch.object( + agent_record, "RECORD_ANCHOR_BASELINE", baseline + ): + rc = agent_record.write_record_anchor_baseline(result) + self.assertEqual(rc, 1, err.getvalue()) + self.assertIn("REFUS", err.getvalue().upper()) + self.assertEqual(baseline.read_text(encoding="utf-8"), before) + + def test_baseline_matches_the_tree_exactly(self) -> None: + """The committed baseline is the tree's rot, in BOTH directions. + + Only the rot buckets are pinned. `ok` is deliberately absent from the + baseline file -- pinning it would make every change that adds or removes + a citation rewrite one shared file. + """ + result = agent_record.scan_record_anchors() + stored = agent_record.load_record_anchor_baseline() + self.assertEqual( + {b: result.counts[b] for b in agent_record.RECORD_ANCHOR_BUCKETS}, + stored, + agent_record.record_anchor_report(result), + ) + self.assertNotIn("ok", stored) + + def test_a_repair_fails_until_the_baseline_is_lowered(self) -> None: + """A ratchet, not a threshold: banking the improvement is mandatory.""" + result = agent_record.scan_record_anchors() + result.counts["stale"] -= 1 + errors: list[str] = [] + agent_record.check_record_anchors(result, errors) + require(errors, r"record-anchor baseline STALE in bucket 'stale'") + + def test_new_rot_fails_the_gate(self) -> None: + result = agent_record.scan_record_anchors() + result.counts["broken"] += 1 + errors: list[str] = [] + agent_record.check_record_anchors(result, errors) + require(errors, r"RECORD ANCHOR REGRESSION in bucket 'broken'") + + def test_a_baseline_is_never_banked_from_a_tree_with_record_errors(self) -> None: + """`--write-baseline` must not run before the checker finishes. + + The mode returned as soon as it had a number. That return happened + before the `if errors:` gate, so a tree that failed any other record + check could still bank its rot. The banked figure then carried the + authority of a run that never passed. The write now happens after the + gate. + """ + digest = agent_record.RECORD_ANCHOR_BASELINE.read_bytes() + stderr = io.StringIO() + with mock.patch.object( + agent_record, "check_roadmap", side_effect=lambda *a: a[1].append("SYNTHETIC") + ), contextlib.redirect_stderr(stderr), contextlib.redirect_stdout(io.StringIO()): + code = agent_record.main(["--write-baseline"]) + self.assertEqual(code, 1) + self.assertIn("SYNTHETIC", stderr.getvalue()) + self.assertEqual(agent_record.RECORD_ANCHOR_BASELINE.read_bytes(), digest) + + class IssueIndexTableShape(unittest.TestCase): """The index is a TABLE, and until #1033 nothing counted its cells. @@ -1423,7 +1596,11 @@ def capture(paths, errors) -> None: with mock.patch.object(agent_record, "check_table_shapes", capture): with mock.patch.object(sys, "stdout", io.StringIO()): with mock.patch.object(sys, "stderr", io.StringIO()): - agent_record.main() + # `main([])` rather than `main()`: #632 gave the checker + # an argparse front end, and `main(None)` therefore parses + # `sys.argv`, which under a test runner holds the runner's + # own arguments and exits 2. The real call site is unchanged. + agent_record.main([]) return captured def test_check_table_shapes_covers_the_issue_index(self) -> None: diff --git a/tests/scripts/test_check_gate_commands.py b/tests/scripts/test_check_gate_commands.py index a0cccf362..698ceba32 100644 --- a/tests/scripts/test_check_gate_commands.py +++ b/tests/scripts/test_check_gate_commands.py @@ -480,6 +480,40 @@ def test_the_baseline_re_pin_is_load_bearing(self): ) self.assertEqual(runnable, set(gates.RUNNABLE_BASELINE)) + def test_dropping_the_anchor_ratchet_from_the_pin_breaks_it(self): + # MUTATION for the 2026-08-14 re-pin (#632). ENG-RECORD-ANCHOR-RATCHET + # entered the runnable population when the row left SPIKE for ACTIVE, so + # the entry added for it must be what keeps the exact pin agreeing with + # the audit. Remove it and the equality assertion has to go red, which is + # what proves the row was pinned because it entered the population and + # not to quiet a gate. + reduced = set(gates.RUNNABLE_BASELINE) - {"ENG-RECORD-ANCHOR-RATCHET"} + self.assertNotEqual(reduced, set(gates.RUNNABLE_BASELINE)) + runnable = {r["id"] for r in gates.audit() if r["verdict"] == "runnable"} + self.assertNotEqual(runnable, reduced) + self.assertEqual(runnable - reduced, {"ENG-RECORD-ANCHOR-RATCHET"}) + + def test_the_anchor_ratchet_credit_is_its_own_gate(self): + # The credit has to be EARNED, not inherited: unlike the weak credits the + # RUNNABLE_BASELINE header admits to, this row's gate IS the checker its + # spec names, so the credited command is the thing under test. Pin both + # halves -- the row audits runnable, and the invocation its Gates section + # carries is the one that reds on either direction of the ratchet. A spec + # rewritten into prose gates goes red here rather than keeping a credit + # it no longer deserves. + row = "ENG-RECORD-ANCHOR-RATCHET" + self.assertIn(row, gates.RUNNABLE_BASELINE) + record = next(r for r in gates.audit() if r["id"] == row) + self.assertEqual(record["verdict"], "runnable", record) + + section = gates.gates_section( + (ROOT / ".agents/specs/record-anchor-ratchet.md").read_text(encoding="utf-8") + ) + self.assertIsNotNone(section) + commands = gates.runnable_commands(section) + self.assertIn("python3 scripts/check-agent-record.py --report", commands) + self.assertIn("python3 tests/scripts/test_agent_record.py", commands) + def test_eng_docs_site_is_credited_for_real_commands(self): # ENG-DOCS-SITE joined the runnable population on arrival rather than # being parked as gates-no-command, so the credit has to be earned by