fix(BACKEND-TENSTORRENT-HOST-FREE-FORWARD): refresh the stale TT Qwen3-0.6B paged-engine golden pair (#1488, #1508) - #1514
Merged
localai-bot merged 10 commits intoAug 21, 2026
Conversation
…e at the first KV block boundary (mudler#1476) The operator gate found captured replay deterministic-degenerate (word salad from ~generated token 30) while host-free eager stayed coherent, reproduced at the landed SHA in clean builds. Two root causes, both fixed: **The RAC page_table was `[C,1]`.** The tt-metal dataflow reader resolves `page_table_ptr[update_idx / block_size]` (`reader_update_cache_interleaved_start_id.cpp:110-111`) — it walks the STICK — so the moment `cur_pos` crossed `block_size` (32) the kernel indexed past the one-column tensor and wrote KV into a garbage physical block. The device tensor now carries the user's whole block-table row `[C, block_table_cols]`; ANY width change (`!=`, block-boundary growth or the multi-request shrink when the longest request finishes — the old `>` let the else-branch copy_to_device TT_FATAL on a shape mismatch) reallocates and RETIRES (keeps alive) the superseded tensor, because a freed device buffer can hand its address to a new allocation while a recorded trace still addresses it; steady state refreshes content only on change (`ptv != e.pt_host`), so there are zero copies inside a block. This lands the "Phase 2 full" refresh the old comment owed and never implemented. **`WarmDecodePos` keyed its skip on `GraphCapturesDone()`.** That counter is process-global and `Reset()` never clears it, so the cold eager step after a boundary reset ran no plus_one and the RE-captured trace read `cur_pos` one position behind. The regime flag now comes from the driver (`s.graph.captured()`); every cold/warm/capture step re-seeds `cur_pos = seq_lens - 1`, and only true replays leave it to the captured plus_one. | Check | Result | |---|---| | captured vs host-free eager, 80 tokens | **NOT byte-identical** — 284B md5 `3b5a579d…` vs 286B md5 `f5ffdf6a…`, first diff byte 174, both arms coherent. Adjudicated per-step top-2 (`VT_TT_DUMP_KV`): 45/80 steps argmax-identical with top-2 values agreeing to ≤0.5 logits; the FIRST divergence (step 46) is a swapped top-2 near-tie — gaps 0.25 vs 0.125 nats (1-2 bf16 ULP), the band this repo tracks for Qwen3-0.6B on TT (mudler#1488 owes the teacher-forced re-adjudication); the 34 later differences are prefix divergence, not numeric evidence | | captured, 160 tokens | coherent, the 80-token answer a strict byte-prefix, 5 boundaries crossed, exit 0 | | captured + `VT_TT_RECAPTURE_EVERY=8` | 9 captures / 71 replays (8 mid-generation re-captures): byte-identical to the plain captured arm, same single step-46 near-tie vs eager. Without this arm the fixed-width gate never fires a `Reset()` (engine preallocates bt_cols=256), so the re-seed guarantee was previously undetected | | mutation reds (/tmp clone, same build config) | G2 (`GraphCapturesDone()>0` early-return restored): steps 1-10 agree, RED at step 11 — non-tie divergence (argmax 220 gap 1.125 vs eager 13), text degrades. G1 (page_table `[C,1]`, no steady refresh): 32 steps agree, RED at step 33 (first step past block_size) with the word salad. Both restores sha256-verified, rebuilt, rerun green with answers byte-identical to the gate runs | | `test_tenstorrent_backend` | 23/23 cases, 831/831 assertions, with AND without an ambient `VT_TT_HOST_FREE_DECODE` | | `test_qwen3_paged_engine` | unchanged from pre-fix (default path; stale golden mudler#1488) | An earlier draft of this commit claimed captured==eager byte-identical over 80 tokens; that was a `grep -m1 '^ Answer'` first-line artifact and is corrected here in the message, the spec, STATUS, and the mudler#1476 index row. - The RAC width-realloc condition is `!=` (shrink included), matching the driver's `cols_changed !=` reset. - `CopyDeviceDeviceIfCapture` / `MemsetDeviceIfCapture` read `VT_TT_HOST_FREE_DECODE` live instead of caching it in a static: the inertness-guard case unsets the env mid-process and must observe the decline, and a suite run under an ambient flag must not pin armed behavior for cases that unset it. The kRopeNeox bit-exact case now unsets the flag itself (mirroring the inertness case), so the suite is green under an ambient flag (was 21/23). - `WarmRacIdx` builds its host page-table tensor only on the copy path and the never-read `RacIdxEntry::idx_host` field/write is gone. - `[TT-DUMP-LOGITS]` carries a top-2 (id, value, gap) suffix under `VT_TT_DUMP_KV` — the near-tie adjudication instrument. - `test_tenstorrent_backend` exits 139 AFTER a green doctest summary (static `optional<ttnn::Tensor>` destruction after device close). Proven pre-existing by A/B stash/build/run at `origin/main`: mudler#1486. - `test_release_metadata` reds on every aarch64 host (fixture stages the host `/bin/true` into an x86_64-named archive), so preflight cannot go green on the TT dev fleet: mudler#1487. It is the one gate that stays red in this change's preflight. - The TT `test_qwen3_paged_engine` golden is stale (anchor drift prompt[1] tok=10, identical before and after this fix — that test runs the default path): mudler#1488 owes the `VT_DUMP_IDS` re-adjudication. Records: spec `## Owed`/`## Now` (including the `retired_pts` defense-in-depth reachability caveat), the STATUS cell, and the corrected mudler#1476 issue-index row. Closes mudler#1476. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
…3-0.6B paged-engine golden pair after oracle re-adjudication (mudler#1488) The committed Tenstorrent anchor for qwen3_greedy_0_6b encoded tokens the current default decode path no longer produces: REQUIRE(anchor_ok) red at prompt[1] tok=10 (engine 14126, committed 6290 — the logged 62901 is a print artifact, mudler#1508). The golden was captured 2026-08-10 under the then-current default path, so the tokens drifted when that path changed, not because the engine diverged. Re-derivation (2026-08-20, Blackhole P150, fix-applied tree 0c3788b): VT_DUMP_IDS captured the full [16,16] battery twice, eager (VLLM_CPP_CUDAGRAPH=0) and captured, byte-identical dumps (md5 b5307e3396d2c5121946c0b746ba5acc) — the mudler#1476 captured/eager fidelity holds across the whole battery, not only the focused pair. The refreshed pair differs from the old golden on 53 cells across 7 prompts: six prompts diverge at exactly one token (p1 tok10, p5 tok10, p10 tok5, p11 tok4, p12 tok13, p15 tok12) and their greedy continuations then follow the new prefix, and prompt 7 is rewritten from tok0 — the new p7 row matches the vLLM greedy sequence exactly, where the old anchor matched neither vLLM nor CUDA. scripts/qwen3-neartie-gap-transformers.py (transformers 4.57.1, torch 2.10.0a0+cpu) teacher-forced the oracle on the new sequence: max gap 375 mnats, zero cells above the 500-mnat band, zero outside-top-K — every refreshed token is a legitimate bf16 near-tie resolution, so the anchor is exact-deterministic and the gap golden is re-derived, not widened. The refreshed pair is derived from the fix-applied tree's TT output and this change therefore stacks on row/BACKEND-TENSTORRENT-HOST-FREE-F1476. The earlier before/after-#1476 comparison recorded in mudler#1488 established only that the p1-tok-10 engine token matched in both arms (exit codes were compared, not the battery); pre-fix TT behavior at the other refreshed cells was never recorded, so a main-based branch could not be gated. Closes mudler#1488. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
…ft values in test_qwen3_paged_engine (mudler#1508) The pinned doctest MessageBuilder stream has no const char* overload, so every separately-bound const char* streamed through MESSAGE or REQUIRE_MESSAGE falls back to the implicit pointer-to-bool conversion and renders as 1. In this test that garbled the anchor-drift failure into "committed anchor=62901" for a golden holding 6290 (and 96251 for 9625): the trailing device-golden ternary printed its bool as a digit glued to the last numeric field, and label ("qwen3-0.6B") printed as 1 in every message. During mudler#1488 the garbled value was misread as corruption of the loaded golden buffers until a byte-level npy verification cleared them; a 7-line harness against the pinned header reproduces the artifact. label, ids_name and gap_name are now std::string and the message ternaries are wrapped, so the drift line prints the model, the real committed anchor, and the device-appropriate remediation arm. Verified on card: the failing case now logs "qwen3-0.6B anchor drift prompt[0] tok=5 engine=15344 committed anchor=9625 — re-run qwen3-neartie-gap.py to refresh the gap golden" (that arm was itself invisible before — the two arms differed only by the same swallowed string). Closes mudler#1508. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
…olden re-adjudication outcome and index mudler#1508 Moves the stale TT paged-engine golden bullet in the spec's Owed list to its resolved form with the re-derivation evidence (eager/captured dump identity, transformers teacher-forced max gap 375 mnats, zero band or top-K violations), corrects the mudler#1488 record to say the before/after-#1476 comparison covered exit codes and the p1-tok-10 token only, and appends the mudler#1508 index row for the doctest const char* print defect found and fixed in the same flow. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
Collaborator
|
This is currently conflicting with |
Main advanced 22 commits (96 files: SPEC-DFLASH2 W4/W5, BPE quadratic merge, MUSIC3 pricing, ROCM block-size contract, CI red repairs). `.agents/benchmark-record.md` conflicted because both sides appended 2026-08-20 entries: resolved by union, keeping this branch's TT host-free mudler#1476 entry and main's MUSIC3-E2E entry. `.agents/issue-index.md` merged by union with no duplicate row (518 rows after). docs/STATUS.md and docs/BENCHMARKS.md auto-merged; this branch's TT rows are untouched by main's edits. Card gates after the merge (Blackhole P150): test_tenstorrent_backend 23/23 + 831/831, and test_qwen3_paged_engine shows the identical documented mudler#1488 anchor drift (prompt[1] tok=10 engine=14126) with no new drift — the golden refresh rides the stacked mudler#1514. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
…> to test_qwen3_dflash2_gguf (mudler#1595) Merging current main into this branch broke every rebuild on the clang-20 hosts: the SPEC-DFLASH2 GGUF drafter arm landed tests/vllm/models/test_qwen3_dflash2_gguf.cpp calling ::getpid() with no POSIX include, so the TU fails to compile and ninja stops before the TT gate targets can relink. The file is byte-identical to origin/main at e2a9e03, so the defect is main's, inherited by any branch that merges main. The sibling tests include <unistd.h> unguarded (test_kimi_linear_paged.cpp:44, test_loader_unaligned_offsets.cpp:46, test_ltx2_loader.cpp:26), so the one-line include is the established convention here. Closes mudler#1595. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
Main advanced past this branch's base (HF model download TLS arm among others). `.agents/issue-index.md` merged by union with no duplicate row; no other file this branch touches changed on main. Recorded because the stacking base mudler#1498 had not yet merged when main moved again. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
…ORRENT-HOST-FREE-1488 Rebasing the stack: mudler#1498 re-merged origin/main (union-resolving the append-only benchmark record) and added the in-flow <unistd.h> fix for test_qwen3_dflash2_gguf (mudler#1595). This branch takes both; no conflicts — its own files (goldens, test_qwen3_paged_engine.cpp, spec, issue-index row) are untouched by the base's merge. Card gates after this merge (Blackhole P150): test_qwen3_paged_engine 16/16 prompts PASS (strict 11/16, near-tie 5/16, max gap 0.375 nats, doctest SUCCESS); test_tenstorrent_backend 23/23 + 831/831; exit-139 after the green summaries is the pre-existing mudler#1486. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
lu-zero
force-pushed
the
row/BACKEND-TENSTORRENT-HOST-FREE-1488
branch
from
August 21, 2026 14:04
c7b55f4 to
8399d61
Compare
This was referenced Aug 21, 2026
localai-bot
pushed a commit
that referenced
this pull request
Aug 21, 2026
…ed at the first KV block boundary (#1476, #1595) (#1498) fix(BACKEND-TENSTORRENT-HOST-FREE-FORWARD): captured decode degenerate at the first KV block boundary (#1476) The operator gate found captured replay deterministic-degenerate (word salad from ~generated token 30) while host-free eager stayed coherent, reproduced at the landed SHA in clean builds. Two root causes, both fixed: **The RAC page_table was `[C,1]`.** The tt-metal dataflow reader resolves `page_table_ptr[update_idx / block_size]` (`reader_update_cache_interleaved_start_id.cpp:110-111`) — it walks the STICK — so the moment `cur_pos` crossed `block_size` (32) the kernel indexed past the one-column tensor and wrote KV into a garbage physical block. The device tensor now carries the user's whole block-table row `[C, block_table_cols]`; ANY width change (`!=`, block-boundary growth or the multi-request shrink when the longest request finishes — the old `>` let the else-branch copy_to_device TT_FATAL on a shape mismatch) reallocates and RETIRES (keeps alive) the superseded tensor, because a freed device buffer can hand its address to a new allocation while a recorded trace still addresses it; steady state refreshes content only on change (`ptv != e.pt_host`), so there are zero copies inside a block. This lands the "Phase 2 full" refresh the old comment owed and never implemented. **`WarmDecodePos` keyed its skip on `GraphCapturesDone()`.** That counter is process-global and `Reset()` never clears it, so the cold eager step after a boundary reset ran no plus_one and the RE-captured trace read `cur_pos` one position behind. The regime flag now comes from the driver (`s.graph.captured()`); every cold/warm/capture step re-seeds `cur_pos = seq_lens - 1`, and only true replays leave it to the captured plus_one. ## Verification (P150, TT Release, thalia; FULL-answer compares) | Check | Result | |---|---| | captured vs host-free eager, 80 tokens | **NOT byte-identical** — 284B md5 `3b5a579d…` vs 286B md5 `f5ffdf6a…`, fir... | captured, 160 tokens | coherent, the 80-token answer a strict byte-prefix, 5 boundaries crossed, exit 0 | | captured + `VT_TT_RECAPTURE_EVERY=8` | 9 captures / 71 replays (8 mid-generation re-captures): byte-identical to th... | mutation reds (/tmp clone, same build config) | G2 (`GraphCapturesDone()>0` early-return restored): steps 1-10 agre... | `test_tenstorrent_backend` | 23/23 cases, 831/831 assertions, with AND without an ambient `VT_TT_HOST_FREE_DECODE` | | `test_qwen3_paged_engine` | unchanged from pre-fix (default path; stale golden #1488) | An earlier draft of this commit claimed captured==eager byte-identical over 80 tokens; that was a `grep -m1 '^ Answer'` first-line artifact and is corrected here in the message, the spec, STATUS, and the #1476 index row. ## Repairs from the fresh review, in the same flow - The RAC width-realloc condition is `!=` (shrink included), matching the driver's `cols_changed !=` reset. - `CopyDeviceDeviceIfCapture` / `MemsetDeviceIfCapture` read `VT_TT_HOST_FREE_DECODE` live instead of caching it in a static: the inertness-guard case unsets the env mid-process and must observe the decline, and a suite run under an ambient flag must not pin armed behavior for cases that unset it. The kRopeNeox bit-exact case now unsets the flag itself (mirroring the inertness case), so the suite is green under an ambient flag (was 21/23). - `WarmRacIdx` builds its host page-table tensor only on the copy path and the never-read `RacIdxEntry::idx_host` field/write is gone. - `[TT-DUMP-LOGITS]` carries a top-2 (id, value, gap) suffix under `VT_TT_DUMP_KV` — the near-tie adjudication instrument. ## Found while gating, filed, not caused here - `test_tenstorrent_backend` exits 139 AFTER a green doctest summary (static `optional<ttnn::Tensor>` destruction after device close). Proven pre-existing by A/B stash/build/run at `origin/main`: #1486. - `test_release_metadata` reds on every aarch64 host (fixture stages the host `/bin/true` into an x86_64-named archive), so preflight cannot go green on the TT dev fleet: #1487. It is the one gate that stays red in this change's preflight. - The TT `test_qwen3_paged_engine` golden is stale (anchor drift prompt[1] tok=10, identical before and after this fix — that test runs the default path): #1488 owes the `VT_DUMP_IDS` re-adjudication. Records: spec `## Owed`/`## Now` (including the `retired_pts` defense-in-depth reachability caveat), the STATUS cell, and the corrected #1476 issue-index row. Closes #1476. ## Operator gate (rerun at this SHA, 2026-08-20) GATE1-captured and GATE1b-eager both exit 0 and reproduce the implementer and re-review runs byte-for-byte (captured 284B md5 3b5a579d8..., eager 286B md5 f5ffdf6aa..., first divergence the adjudicated step-46 near-tie flip; both TT-confirmed by UMD lines and `Asynchronous scheduling is disabled`). The #1476 degeneration is gone on the operator's own run. GATE2 (paged-engine) exit 8 is the unchanged pre-existing #1488 signature (anchor drift prompt[1] tok=10 engine=14126) — pending that re-adjudication, not a regression. ## Review record Fresh review of the first draft FAILED (the byte-identical claim was a first-line-only comparison artifact); a fresh implementer repaired all six findings and a fresh scoped re-review of 2b06f98 returned PASS with three note-level items (two stale comments to reword on next touch, one BENCHMARKS citation to fold into the operator rerun). Inherited reds, all pre-existing at origin/main and filed: #1486 (suite exit-139 after green summary), #1487 (aarch64 preflight red — the one gate that stays red here), #1488 (paged-engine golden re-adjudication). The operator gate rerun on card remains the row's ## Update (2026-08-21): main re-merge + an in-flow main-side fix Main moved 22 commits (96 files); this branch re-merged `origin/main` at `e2a9e035d` (union-resolving the append-only `.agents/benchmark-record.md`) and gained one more commit: - **`75d8c8b76` — missing `<unistd.h>` in `test_qwen3_dflash2_gguf.cpp` (#1595).** The SPEC-DFLASH2 GGUF arm landed on main calling `::getpid()` with no POSIX include, so the TU does not compile under clang-20 and every post-merge rebuild on a clang host stopped before the TT gate targets could relink. One-line include, matching the sibling-test convention; found and fixed in-flow, issue #1595. Post-merge card gates (Blackhole P150, head `75d8c8b76`): `test_tenstorrent_backend` 23/23 + 831/831 (exit-139 after the summary is the pre-existing #1486); `test_qwen3_paged_engine` shows the identical documented #1488 anchor drift (`prompt[1] tok=10 engine=14126`) and no new drift — the golden refresh and the doctest print fix ride the stacked #1514. `agent-preflight` is fully green on this head (main's `GATE-CI-RED-REPAIR` cleared the earlier standing reds #1487/#1504). Closes #1476. Closes #1595. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:zai-glm-5.3 [Maki]
mudler#1498 landed as a squash, so its commits on this branch no longer share an ancestor with main and the spec file conflicted. Resolved by keeping this branch's text for both hunks: main carries the superseded "still Owed" wording for mudler#1488 and mudler#1508, including the 62901 anchor value that mudler#1508 proved was a print artifact of the 6290 the golden actually holds. Every other path merged without conflict. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [Claude Code]
mudler#1498 landed as a squash, so its commits on this branch no longer share an ancestor with main and the spec file conflicted. Resolved by keeping this branch's text for both hunks: main carries the superseded "still Owed" wording for mudler#1488 and mudler#1508, including the 62901 anchor value that mudler#1508 proved was a print artifact of the 6290 the golden actually holds. Every other path merged without conflict. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [Claude Code]
localai-bot
pushed a commit
that referenced
this pull request
Aug 21, 2026
… tree that will land Brings the branch from `f4ccabbb4` to `2e7f3bee7`, thirteen commits, so the gate runs on the tree that will land rather than on the one the wave opened. Two of them change what this row's gate reads: `13548db8f` (`SPEC-DFLASH2`, #1575, #1581) moves the GGUF DFlash2 test onto the `process_id` seam and `369a0c2aa` (`SPEC-DFLASH2`, #1617) trims its include, and both sit in the speculative-decoding suite this row extends. `b80659276` (`BACKEND-ROCM`, #1046) and `49c64bbc8` (`BACKEND-TENSTORRENT-HOST-FREE-FORWARD`, #1514) each add CTest entries, so the pre-merge tree runs a smaller suite than the one that lands. NO CONFLICT, and the clean merge was still verified rather than trusted. Five files are touched by both sides, and for each the merged bytes were read: the merged file differs from `2e7f3bee7` by exactly this row's edit and differs from `e1cd772f1` by exactly main's edit. `.agents/issue-index.md` is append-only under `merge=union` and was re-derived by COUNTING the merged rows, not by arithmetic on either side's number: 520 rows at the merge base, 534 on main, 525 here, 539 merged, which is main's 534 plus this row's 5. Zero deletions, zero modified rows, no duplicate issue number, every row of both parents present byte-for-byte, and `git diff --unified=0` against `2e7f3bee7` removes zero lines. The count was checked because the same driver silently DROPPED main's last row on the sibling `row/SPEC-DFLASH2-W6` merge taken from the same main SHA; it did not do so here, and nothing but a row count would have said which. `docs/STATUS.md` had both sides edit a row and the keys are disjoint: main rewrote `Tenstorrent Blackhole`, this row rewrote `Speculative decoding`. `docs/BENCHMARKS.md` is main's file plus this row's single appended `SPEC-DRAFTER-CHAIN` line, with main's rewritten host-free-decode row intact. `docs/USAGE.md` and `tests/CMakeLists.txt` interleave additions only. `.agents/engine-matrix.md` is the one keyed record main did not touch in this range, so this row's version stands unopposed. Its summary counters were still re-derived by counting the merged section rows with the repository's own parser (`scripts/check-agent-record.py` `parse_claim_rows`): 169 rows, ANCHOR-BACKFILL 35, PARTIAL 18, SPIKE 4, READY 12, ACTIVE 41, GATING 8, DONE 12, INVENTORIED 38, BLOCKED 1, which is what the `**Total**` row states. BLOCKED has no column, which is why the row count exceeds the column sum by one. Rows: SPEC-DRAFTER-CHAIN-W1, GATE-QWEN38-27B-FP8-BLOCK, VT-REFTIER-HOST-ADDRESSABLE, ENV-LEASE-GPU-CAPABILITY, SERVE-C-ABI, BACKEND-ROCM, GATE-DOC-CHECKPOINT-STATES, SPEC-DFLASH2, BACKEND-TENSTORRENT-HOST-FREE-FORWARD FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: ClaudeCode:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge order — read first
This change is stacked on #1498 (its golden is derived from that tree), but the stacking branch exists only on the fork, so this PR is based on
mainand currently also carries #1498's commit0c3788b9c. Merge #1498 first; this PR then shows only the three commits below. Do not merge this before #1498.What
Three commits closing two issues:
qwen3_greedy_0_6bencoded tokens the current default decode path no longer produces (REQUIRE(anchor_ok)red at prompt[1] tok=10, engine 14126 vs committed 6290). The golden was captured 2026-08-10 under the then-current default path; the tokens drifted when that path changed, not because the engine diverged.MessageBuilderhas noconst char*overload, so every separately-streamedconst char*renders as bool1: the anchor-drift message printedcommitted anchor=62901for a golden holding6290(and96251for9625), andlabelprinted as1in every message. During test_qwen3_paged_engine TT golden is stale: anchor drift prompt[1] tok=10 (engine=14126, committed=62901), unchanged by the #1476 fix #1488 the garbled value was misread as golden-buffer corruption until byte-level npy verification cleared the buffers.label/ids_name/gap_nameare nowstd::stringand message ternaries are wrapped.Re-derivation evidence (#1488)
VT_DUMP_IDScaptured the full [16,16] battery twice on Blackhole P150 from the fix-applied tree0c3788b9c— eager (VLLM_CPP_CUDAGRAPH=0) and captured dumps byte-identical (md5b5307e3396d2c5121946c0b746ba5acc): the TT host-free captured decode goes degenerate at the first KV block boundary; token-exactness evidence does not reproduce #1476 captured/eager fidelity holds across the whole battery.scripts/qwen3-neartie-gap-transformers.py(transformers 4.57.1, torch 2.10.0a0+cpu) teacher-forced the oracle on the new sequence: max gap 375 mnats, zero cells above the 500-mnat band, zero outside-top-K — the anchor is exact-deterministic and the gap golden is re-derived, not widened.Why it needs #1498
The refreshed pair is derived from the fix-applied tree's TT output. The before/after-#1476 comparison recorded in #1488 compared exit codes and the p1-tok-10 token only; pre-fix TT behavior at the other refreshed cells was never recorded, so a main-based branch could not be gated.
Gates (Blackhole P150, head
4496116e8)test_qwen3_paged_engine: 16/16 prompts PASS (strict token-exact 11/16, near-tie-band 5/16, max gap 0.375 nats @ p15 t12, 0 forward-divergent), BACKEND PROOF device type 6 with 0 declines, UMD lines present. Exit 139 after the green summary is the pre-existing test_tenstorrent_backend SIGSEGVs after a fully green doctest summary: static ttnn::Tensor caches destroyed after the device closes #1486 static-destruction segfault.test_tenstorrent_backend: 23/23 cases, 831/831 assertions (same known exit-139).Update (2026-08-21): rebased onto the updated #1498 head
Main moved 22 commits; the stacking base re-merged
origin/mainate2a9e035dplus thein-flow
getpidinclude fix (#1595, riding #1498). This branch merged that head(
c7b55f4dd, no conflicts). Card gates on the new head:test_qwen3_paged_engine16/16 prompts PASS (strict 11/16, near-tie 5/16, max gap 0.375 nats, truthful
qwen3-0.6Blabels, UMD lines present, doctest SUCCESS);test_tenstorrent_backend23/23 + 831/831 (exit-139 = pre-existing #1486).
agent-preflightfully green.Closes #1488. Closes #1508.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:zai-glm-5.3 [Maki]