fix(VT-FP8-QUANT-ARCH-GATE): register QuantFp8Static from an unconditional CUDA TU (#960)

[localai-bot]

Closes #960. Closes #989.

Spec: .agents/specs/vt-fp8-quant-arch-gate.md.
Owning row: MODEL-TEXT-nemotron-h-nemotron-hfor-causal-lm (#517), whose A2-Q1 unit (#810) is the caller this blocks. Base: #940.

What was wrong

vt::QuantFp8Static's only CUDA registration was src/vt/cuda/cuda_matmul_fp8_cutlass.cu:376, and CMakeLists.txt:1668 compiles that translation unit only when VT_CUTLASS_FP8_ARCHS is non-empty.

The kernel body has no cutlass dependency of any kind — zero cutlass/CUTLASS tokens in QuantFp8StaticKernelCuda (:353-370). It is out[i] = e4m3_rne_sat(x[i] * (1/input_scale)), a grid-stride elementwise loop over a hardware convert intrinsic. It shared a TU with the cutlass sm120 fp8 GEMM and inherited that GEMM's arch set.

On sm_110 (Thor) that intersection is empty. cutlass-fp8: DISABLED (no requested arch in [110] provides it) is the arch's documented normal profile, not a misconfiguration. So OpId::kQuantFp8Static was not registered for DeviceType::kCUDA at all, on that arch and on every CUDA arch outside the cutlass-fp8 cell.

Nothing refused first, which is what made it expensive. The op's GEMM partner kMatmulFp8CublasLt is registered unconditionally (src/vt/cuda/cuda_matmul.cu:920), so MatmulFp8CutlassD's guard — which keys on it — passed. The missing quant then resolved through src/vt/op_provider.cpp:501 to the portable CPU reference tier, eligible because CudaBackend::UnifiedMemory() is true, which dereferenced device pointers on the host:

[vt reference-tier] op=QuantFp8Static device=cuda has NO native kernel; running the PORTABLE CPU fallback (correct but slow)
SIGSEGV

Nothing silently dequantized. Nothing refused either. It crashed one call later, under a banner claiming "correct but slow".

The change

src/vt/cuda/cuda_quant_fp8.cu	new, unconditionally compiled TU; kernel + helpers + registration moved verbatim
CMakeLists.txt:1586	added to target_sources(vllm PRIVATE ...) directly inside if(VLLM_CPP_CUDA)
src/vt/cuda/cuda_matmul_fp8_cutlass.cu:322	kernel and registration removed; a comment records why, so the next reader does not move it back

The only edit to the moved code is the local Check() message prefix, matmul_fp8_cutlass → quant_fp8, which stopped being true the moment the code moved. No arithmetic, no dtype dispatch, no dispatch condition is touched. Because the new TU is not in _VT_CUDA_FEATURE_SOURCES, CMakeLists.txt:2231-2236 gives it the full ${VLLM_CPP_CUDA_ARCHITECTURES} gencode list, which is the point.

Why a new file, not cuda_matmul.cu or cuda_ops.cu. Both were real candidates and both were rejected in the spec. cuda_matmul.cu already hosts the unconditional fp8 GEMM registrations — the strongest argument for it — but it is the cuBLAS/cuBLASLt GEMM wrapper TU and this is not a GEMM. cuda_ops.cu is unconditional, already includes <cuda_fp8.h>, and hosts RmsNormQuantFp8, literally the fused arm of this same math. The defect, though, is precisely that a kernel's compilation was governed by a feature it does not use, and both alternatives re-create a weaker form of that by binding it to an unrelated file's includes and requirements. A file named for the op makes the invariant readable in the CMake diff, is what the structural checker can assert without inference, and is the natural home as the fp8 activation-quant family grows.

This also restores upstream's own partition: vLLM builds static_scaled_fp8_quant from csrc/quantization/w8a8/fp8/common.cu in the unconditional VLLM_EXT_SRC list, and gates only its cutlass scaled_mm sources behind CUDA_ARCHS intersections.

Two instruments, because neither covers the other

G4, tests/vt/test_ops_fp8_cpu.cpp — the runtime pin, and the stronger claim: it observes the property that matters, OpRegistered(kQuantFp8Static, kCUDA), rather than a proxy. It needs no CUDA device (registration is a table fill before main), because "which build" is the axis the defect lived on. Its second assertion requires kMatmulFp8Cutlass to track VT_CUTLASS_FP8 instead, so the case proves the two are independent rather than merely that one is present.

scripts/check-cuda-op-arch-gate.py — the tripwire. G4 can only speak on a CUDA build without cutlass-fp8, and no CI job produces one: the GB10 gate host resolves cutlass-fp8: ENABLED, where the defect is unreachable by construction, and every other job is CPU-only. G4 would not have caught #960 before it landed; it caught it here only because someone carried a binary to Thor. The checker reads the build description, runs in the ordinary checker lane on every host, and fails at PR time. Four clauses per entry, no inference about what a kernel "needs": HOME (the TU is in the unconditional list), REGISTERED (exactly one live registration), UNGUARDED (preprocessor depth 0, so an #ifdef VT_CUTLASS_FP8 wrapper is not a pass), EXCLUSIVE (no second kCUDA registration elsewhere). C++ is read through checker_text.normalize_source, so a commented-out or #if 0-ed registration reads as absent.

Evidence

Thor, sm_110, no cutlass — the red-before

CUDA target architectures: 110, cutlass-fp8: DISABLED, CUTLASS not found on every build. BUILD_EXIT=0, warnings: 0, enospc: 0. Disk 319 G → 318 G.

tree	binary sha256	test_ops_fp8_cpu
base 0e1bee42f	6b4d4df071a6…	test cases: 2 | 1 passed | 1 failed | 2 skipped · assertions: 43 | 43 passed | 0 failed · Status: FAILURE! · exit 139 (SIGSEGV)
base + G4 only, RED-first	63b7940e8609…	G4 isolated: test cases: 1 | 0 passed | 1 failed | 4 skipped · assertions: 2 | 1 passed | 1 failed · Status: FAILURE! · exit 1
base + G4 + fix	690bf71448ea…	test cases: 5 | 5 passed | 0 failed | 0 skipped · assertions: 62 | 62 passed | 0 failed · Status: SUCCESS! · exit 0
HEAD, fresh tree, clean build	dc83e683f4fd…	identical: 5 | 5 passed | 0 failed · 62 | 62 passed | 0 failed · SUCCESS! · exit 0

The base row reproduces #960 verbatim, including the trap #844 names: assertions: 43 | 43 passed | 0 failed printed beside Status: FAILURE!, so anything grepping the assertions line alone reads a crash as green.

The RED-first row is the one that matters. G4's first assertion failed — CHECK( vt::OpRegistered(vt::OpId::kQuantFp8Static, DeviceType::kCUDA) ) → values: CHECK( false ) — while its second passed, CHECK_FALSE( false ) on kMatmulFp8Cutlass, so the case was not vacuous and the arch genuinely lacks the cutlass GEMM. Non-zero case count in both directions. The case name carries no comma: -tc splits on commas, and a comma would have selected nothing and reported SUCCESS! with exit 0.

In the green rows, G2 — the CPU-vs-CUDA byte comparison, zero tolerance, five scales × 4096 elements — executes on sm_110 for the first time and passes, bad == 0 throughout, with no [vt reference-tier] banner printed at all. That closes the arm vt-fp8-w8a8-cpu-arm.md (#468) recorded as owed on a non-cutlass CUDA arch. The last row is a fresh git archive into a new directory with no build state, because three incremental rebuilds of one tree do not prove the committed tree builds.

GB10, sm_121a — unchanged, which is the claim

Both builds assert CUDA target architectures: 121a and CUDA feature cutlass-fp8: ENABLED for [121a] in the configure log; a DISABLED line would void the result. BUILD_EXIT=0, warnings: 0, enospc: 0, -j 4, 2.7 T free. Every binary sha differs between columns, so neither is a stale artifact.

suite	BEFORE (0e1bee42f)	AFTER
test_ops_fp8_cpu	4 | 4 passed | 0 failed · 60 | 60 passed | 0 failed · SUCCESS!	5 | 5 passed | 0 failed · 62 | 62 passed | 0 failed · SUCCESS!
test_ops_fp8_cutlass	8 | 8 passed | 0 failed · 86 | 86 passed | 0 failed · SUCCESS!	identical
test_linear_method	10 | 9 passed | 1 failed · 97 | 95 passed | 2 failed · FAILURE!	identical
test_ops_fused_chain	10 | 10 passed | 0 failed · 583 | 583 passed | 0 failed · SUCCESS!	identical

The only delta is test_ops_fp8_cpu gaining exactly G4: +1 case, +2 assertions. The four pre-existing cases and their 60 assertions are untouched.

test_linear_method fails identically in both columns, which is how this run proves it is not ours rather than assuming it: linear_method: MXFP4 fused gate_up ~= split (numerically) + fused path ran, test_linear_method.cpp:247, CHECK( after == before + 1 ), twice — a Marlin dispatch counter on a suite with no fp8 arm. That is #907's test_linear_method row at the cutlass-enabled build's shape.

One trap worth reading, because it nearly produced a false result. The first AFTER build reported 4 cases / 60 assertions — G4 missing from a binary whose sha had changed. tar restored the test file with its local mtime (06:46 UTC), older than the object compiled during the base build (07:10 UTC), so ninja skipped the compile and relinked only because libvllm.a had changed. The suite reported on a binary containing the fix but not the test. The table above is from a rebuild after touch. Verify the case count, not the sha.

CPU-only, local

Release-equivalent build, BUILD_EXIT=0, 0 warnings. ctest 489/489 passed, 0 failed (2 skipped for absent checkpoints). test_ops_fp8_cpu reads 4 / 56 here: G4 compiles out on a non-CUDA build, which is correct and is why it is #if defined(VLLM_CPP_CUDA).

scripts/agent-preflight.sh: every gate green except test_cpu_x86_llamacpp_floor, which exited NO_QUIET_WINDOW (4) at loadavg 77.49 while this box was building — the harness refusing to measure under contention, #618, inherited.

The checker's mutation evidence, executed rather than described

scripts/check-pr-size.py --base <merge-base> --head HEAD passes, which is not a formality: its evidence contract checks the branch into a scratch worktree, runs tests/scripts/test_check_cuda_op_arch_gate.py at HEAD (must pass, non-zero case count), then overwrites the checker with a disabled stub and re-runs the same module (must fail, non-zero case count). Both halves are machine-verified. Locally the suite is 14 cases: one per clause, the two "text the compiler never sees" disguises, a live-tree case, and test_empty_source_list_is_not_a_pass — because a checker reporting OK on a parse that matched nothing is a green light attached to no measurement.

That contract also caught a defect in the first version of the suite: it bound the checker's functions at import time, so the stub produced an ImportError rather than failing cases and the contract reported semantic evidence did not execute tests — neither red nor green, the instrument declining to say. Fixed before this branch was pushed.

#844 is not fixed here

This removes one live instance and does not address the class. The portable reference tier still accepts DeviceType::kCUDA tensors, still dereferences them, and still calls itself "correct but slow" while doing it. The next feature-gated op to lose its native kernel reproduces #960 exactly. That repair is a larger change to the tier and wants its own spec; #844 stays open.

#989 fixed in flow, because it had to be

Registering the new checker's creation mutation means editing scripts/check-pr-size.py, and that file's own evidence contract requires its suite green at HEAD. tests/scripts/test_check_pr_size.py has been red on main since #888 added .agents/reachability.md with no path class — and red silently: it runs in no CI job and no preflight suite, so the only thing that ever loads it is the checker-evidence contract, which fires only when someone edits a checker. Third instance of the class after #856 and #668, fixed the same way both were: one entry in PROCEDURE_FILES, next to its sibling guides, with a comment naming the issue. Wiring that suite into CI is deliberately not done here.

Why this replaces #990

#990 was the same change on row/VT-FP8-QUANT-ARCH-GATE-960 and is closed, not
abandoned. documentation-checkpoint is a PER-COMMIT gate and it was right: the
commit that edits CMakeLists.txt owes docs/USAGE.md in that same commit, and
mine did not. There is an honest thing to say there, so it is said — a DISABLED
CUDA feature removes its kernels, not the ops that do not need them, and a
[vt reference-tier] banner naming an op on a cuda device is a defect to
report rather than a slow path to accept.

A later commit cannot satisfy a per-commit gate, so the history had to be
rebuilt. main is never force-pushed and neither is a row branch here, so this
is a new branch with the corrected history rather than a rewrite of the pushed
one. The trees differ by exactly docs/USAGE.md, +25 lines
(git diff --name-only between the two heads). Every measurement above was taken
against that tree and none of it is restated from memory.

Not in scope

The kernel's arithmetic, option parsing and dispatch conditions (all moved byte-for-byte); MatmulFp8Cutlass, whose arch gate is correct because it really is a cutlass sm120 kernel; and wiring NemotronH to anything (#810 / #517 A2-Q1), which this unblocks but does not do.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]

[localai-bot]

CI at b05fd38a7: every job green except the two inherited MSVC lanes

Recorded here because the branch has since moved and that run's verdict would otherwise be lost. The commits pushed on top of it are .agents/specs/vt-fp8-quant-arch-gate.md, a merge of origin/main at 3ce1cf7c7, and an .agents/issue-index.md order repair — no source, test, CMake or checker file differs.

job	result
agent-record · build · build-test-cpu · build-test-cpu-arm64 · build-test-vulkan	pass
commit-protocol-tag · cuda-arch-features · cuda-fat-build · device-leakage	pass
documentation-checkpoint · drift-check · last-gated-commit · plan · pr-size	pass
sanitize-cpu (address,undefined) · sanitize-cpu (thread) · vulkan-spirv-freshness	pass
verify (cpu, …) · verify (vulkan, …)	pass
windows-msvc-cpu · windows-msvc-vulkan	fail — inherited, see below

pr-size passing is the load-bearing one: its evidence contract runs tests/scripts/test_check_cuda_op_arch_gate.py at HEAD, then replaces the new checker with a disabled stub and re-runs the same module, and requires pass-then-fail with a non-zero case count on both halves. The checker's red-before/green-after is therefore machine-verified in CI, not asserted in this description.

The MSVC red, named rather than labelled

ltx2_video.cpp
…\MSVC\14.44.35207\include\vector(1461,29): warning C4244: '=': conversion from 'const double' to 'float', possible loss of data
  (compiling source file '../src/vllm/multimodal/ltx2_video.cpp')
…\include\vector(1461,29): error C2220: the following warning is treated as an error

That is #968 — the narrowing positions.assign sites #964 added — surfacing from inside MSVC's <vector> under /WX. A fix is already open as #983. main carries no baseline for these jobs because #584 makes windows-msvc-* PR-only, which is exactly why it presents to each author in turn as their own diff's fault.

Matched arm, measured not assumed: open PRs #988 and #982 fail at the byte-identical step name, Build and execute the native Windows CPU focused gate, and neither shares a file with this row. This branch touches no src/vllm/multimodal/ path at all.

Post-merge re-verification at 142fcba7f

origin/main moved four commits (#993, #982, #988, #992) while this row was measuring. Merged at the pinned SHA 3ce1cf7c7 per #841, rebuilt (BUILD_EXIT=0, 0 warnings) and re-ran the full local gate: 491/491 passed, 0 failed — 491 rather than 489 because that merge adds two suites — plus check-pr-size, both checker suites, and the record gates against the new base.

One of those four is worth naming: #988 split #810's A2-Q into A2-Q1 and A2-Q2, and A2-Q1 is the unit this row exists to unblock.

The merge produced a clean but WRONG .agents/issue-index.md and it is repaired in 142fcba7f's parent. Every row from both sides survived and check-issue-index-append-only.py passed, but merge=union interleaves by hunk rather than by arrival, so main's newest row (#987) landed after this branch's #960 and #989. "No conflict" says nothing about order, and order is the file's entire contract. Caught by the check the union driver cannot make — is origin/main's file a strict prefix of ours? — and repaired by taking main's copy wholesale and re-appending only this branch's two rows, verified byte-for-byte identical across the shared 251 rows.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]