From f28479ef16c00f9cf26d2a0aa74328d96f8eac09 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Fri, 21 Aug 2026 20:20:50 +0000 Subject: [PATCH 1/4] record(VT-REFTIER-HOST-ADDRESSABLE): the test cited as pinning CUDA reads the CPU backend #1620 recorded that `tests/vllm/platforms/test_platform.cpp` pins the CUDA backend's `DeviceMemoryIsHostAddressable()` default. It does not. `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` at `:284`, but its `backend()` returns `vt::GetBackend(DeviceType::kCPU)` at `:285`, so the `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` at `:329` asserts against the CPU backend. The fixture's own comment says so. The conclusion survives and the code is unchanged: `CudaBackend` declares no override, so it inherits the base `false` at `include/vt/backend.h:77`. But it holds by ABSENCE of an override, which is a weaker claim than a pin, and the record read as though a test stood behind it. Nothing in the tree pins the real `CudaBackend`. No gate could have caught this. The assertion passes, and it passes for a reason unrelated to what it claims to measure -- the same shape as a mutation that never applied reading as a passing test. The wrong citation landed in three places and one of them is permanent: `docs/ENVIRONMENT.md:210` is corrected here, the commit message stands as what was believed at the time, and `.agents/issue-index.md:552` is append-only and can never be edited. That last one is why this is a new row rather than an edit, and why a record-only change is the right shape: the record IS the work. Pinning the real `CudaBackend` needs a CUDA device and stays owed on the row. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 1 + .agents/specs/vt-reference-tier-host-addressable.md | 8 ++++++++ docs/ENVIRONMENT.md | 2 +- 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 3c077a974..cb529d5c9 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -550,3 +550,4 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1613](https://github.com/mudler/vllm.cpp/issues/1613) | `GATE-QWEN38-27B-FP8-BLOCK` | **The `Qwen/Qwen3.8-27B-FP8` block-wise token gate cannot be taken, because the 28.75 GiB checkpoint is not on the share.** `/mnt/nas_share/rc/ckpt/` holds `qwen3.8-27b-hf`, which is the **bf16** artifact -- no `quantization_config` key, `text_config.dtype = bfloat16` -- and `qwen3.8-q1_0`. Neither is this subject. The share has 3.4 TiB free, so the cost is AUTHORITY: `.agents/developer-preferences.md` authorizes large downloads for the `SPEC-DFLASH2` assets only. Nothing else blocks the gate, and that was not known before: a range-request audit of all 66 shard headers at revision `017b9c7a` shows every one of the 407 `F8_E4M3` tensors has `N % 128 == 0` and `K % 128 == 0`, so the sm120 complete-scale-block refusal (#1453) that makes DSV3's `kv_a_proj_with_mqa` unservable blocks NOTHING here; the ragged GDN `in_proj_a`/`in_proj_b` `[48, 5120]` are `BF16` and named in `modules_to_not_convert`; `weight_scale_inv` ships `BF16` (byte-checked via `data_offsets`, not the label) which `LoadFp8BlockRaw` already widens by value; and the per-layer `layers-.safetensors` naming already resolves through `SelectWeightFiles`. Spec `.agents/specs/gate-qwen38-27b-fp8-block.md`, parent #1189 | gap | | [#1614](https://github.com/mudler/vllm.cpp/issues/1614) | `GATE-QWEN38-27B-FP8-BLOCK` | **Three sites said `Qwen/Qwen3.8-27B-FP8` ships "~400" `modules_to_not_convert` entries, and at revision `017b9c7a` it ships 882** (882 unique, 636 outside the vision tower). The number is the evidence for an ARGUMENT -- it is why `IsFp8BlockProjection` reads the config AND the tensors instead of probing dtypes -- so being wrong by more than 2.2x invites the next reader to re-derive it. No reading of the list produces ~400: the visual entries are duplicated under two naming conventions, so distinct modules are about 759, and half of 882 is 441. Sites: the comment above `IsFp8BlockProjection`, the comment above `Fp8BlockQuantConfig::modules_to_not_convert`, and `.agents/specs/model-fp8-block-weight.md`. The routing itself is correct and no defect in it is asserted; two other claims in the same comment were checked against the checkpoint headers and hold (zero `input_scale` tensors, and the `[96, 40]` block-grid hazard is real). Found while auditing the checkpoint for #1613, fixed in the same flow | bug | | [#1502](https://github.com/mudler/vllm.cpp/issues/1502) | `VT-REFTIER-HOST-ADDRESSABLE` | **`docs/ENVIRONMENT.md` described `VT_ADOPT_DEVICE_BYTES` as Vulkan-only and said it has "No effect on CUDA/CPU/Metal", and [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477)) made both halves false.** That change moved `ReferenceTierEligible` off `UnifiedMemory()` onto `Backend::DeviceMemoryIsHostAddressable()` and added truthful overrides so no backend lost the reference tier, so `MetalBackend` now answers `MetalContext::unified_memory()` and `RocmBackend` answers its `unified_memory_`. The weight loader gates the lever on exactly that predicate, at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`, so the lever ACTS on Apple silicon and on an integrated ROCm part. **The correction is not "add two backend names".** Every number in that row is GB10 through Vulkan, and nobody has measured the lever on either new arm, so the row now separates the backends it is MEASURED on from the backends that merely satisfy the predicate — reach and measurement are different claims and the row read as if the measurement covered the reach. CUDA and CPU stay inert and are unchanged: neither overrides the default `false`, which `tests/vllm/platforms/test_platform.cpp` pins for GB10, and the CPU backend answering `UnifiedMemory() == true` while the narrower predicate stays `false` is the whole reason the two properties are separate. The MEASUREMENT on Metal and integrated ROCm stays owed and is listed under `## Owed` in [`vt-reference-tier-host-addressable.md`](specs/vt-reference-tier-host-addressable.md); it needs an Apple-silicon box or an integrated AMD part | documentation | +| [#1635](https://github.com/mudler/vllm.cpp/issues/1635) | `VT-REFTIER-HOST-ADDRESSABLE` | **[#1620](https://github.com/mudler/vllm.cpp/pull/1620) cited `tests/vllm/platforms/test_platform.cpp` as pinning the CUDA backend's `DeviceMemoryIsHostAddressable()` default, and that test reads the CPU backend.** `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` (`:284`) while its `backend()` returns `vt::GetBackend(DeviceType::kCPU)` (`:285`), so the `CHECK_FALSE` at `:329` asserts against CPU and the test's own comment says so. **Nothing in the tree pins the real `CudaBackend`.** The CONCLUSION survives — `CudaBackend` declares no override, so it inherits the base `false` at `include/vt/backend.h:77` — but it holds by absence, which is a weaker claim than a pin and must not read as one. No gate could catch this: the assertion passes, and it passes for a reason unrelated to what it claims to measure. The wrong citation landed in three places and one is PERMANENT: `docs/ENVIRONMENT.md:210` (corrected in this flow), the commit message, and `.agents/issue-index.md:552`, which is append-only and can never be edited — which is why this correction needs a row of its own rather than an edit. Found by post-hoc review of four merged records pull requests, not by a gate. OWED: either pin the real `CudaBackend` or state in the record that the default holds unpinned | bug | diff --git a/.agents/specs/vt-reference-tier-host-addressable.md b/.agents/specs/vt-reference-tier-host-addressable.md index fba496975..4a4a997fb 100644 --- a/.agents/specs/vt-reference-tier-host-addressable.md +++ b/.agents/specs/vt-reference-tier-host-addressable.md @@ -79,6 +79,14 @@ Out of scope, and why: authoring host has no `nvcc` and no GPU. Bundling it here would put a change that only a leased box can verify inside a change that a CPU box verifies completely. Recorded under [`## Owed`](#owed). + +- [#1635](https://github.com/mudler/vllm.cpp/issues/1635) is OPEN and owned here. The record's + claim that `tests/vllm/platforms/test_platform.cpp` pins the CUDA + `DeviceMemoryIsHostAddressable()` default is FALSE: that fixture's `backend()` returns the CPU + backend, so the assertion never reads `CudaBackend` at all. The conclusion survives by absence of + an override; the pin does not exist. `docs/ENVIRONMENT.md` is corrected, but the copy in + `.agents/issue-index.md:552` is append-only and permanent. Pinning the real `CudaBackend` stays + owed and needs a CUDA device to exercise. - **Copy-in and copy-out instead of refusal.** Rejected under [Design](#design). - **Registering a CUDA kernel for `kMatmulFp8BlockScaled` on a CUTLASS-less diff --git a/docs/ENVIRONMENT.md b/docs/ENVIRONMENT.md index 1303a89a7..d9a056d0c 100644 --- a/docs/ENVIRONMENT.md +++ b/docs/ENVIRONMENT.md @@ -207,7 +207,7 @@ portable/reference path. In normal operation leave them unset. | `VT_DFLASH_ATTN_WARP` | off (CUDA) | `=1` falls back to the older per-key warp-reduction block-attention kernel instead of the default chunked reduce-scatter form. Kept for the same-binary A/B that recorded the verdict | | `VT_DFLASH_ATTN_KEYLANE` | off (CUDA) | `=1` selects the one-key-per-lane block-attention form. **MEASURED NEGATIVE and not a tuning knob:** 28.90 s/step against the per-key warp kernel's 18.73 on the same binary (sm_110, MiniMax-H3 512x512/33f, seq 3224), 54% slower, because giving each lane a whole K row makes every K load 32-way scattered. Kept only because it is the experiment that located the real constraint | -| `VT_ADOPT_DEVICE_BYTES` | on (acts only where the backend advertises host-addressable device memory: Vulkan always, Metal and ROCm where the part reports unified memory. MEASURED on Vulkan only) | After a weight is uploaded, re-point its host buffer AT the device allocation instead of keeping a second copy. On a unified box the two copies come out of the same RAM. MEASURED on GB10, Qwen3.6-27B bf16 (50.89 GiB): with the mirror the process reaches **VmRSS 100.759 GiB** and drives the machine to MemAvailable 13.85 / MemFree 1.13 GiB of 119.6 GiB before it has even finished allocating; without it the same load completes at **VmHWM 53.413 GiB**. Qwen3-4B: **16.392 -> 9.607 GiB**. Vulkan allocation is byte-identical either way. `0` is the same-binary A/B back to the two-copy behaviour, and it moves ONLY this lever: the `VT_LOAD_DIRECT_UPLOAD` post-upload source-page release still runs at `0` and on a backend without host-addressable device memory, so the two knobs stay independent. It is an adoption, not a release — the bytes survive at the device address and every reader sees them — so tokens are unchanged either way (`test_opt_paged_engine` on Vulkan is 6/6 token-exact, 96/96, both arms). **REACH AND MEASUREMENT ARE NOT THE SAME SET**, and this row used to read as if they were. The lever is gated on `vt::Backend::DeviceMemoryIsHostAddressable()`, read at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`. That predicate defaults to `false` (`include/vt/backend.h`), so a backend has to opt in, and THREE now do. `VulkanBackend` (`src/vt/vulkan/vulkan_backend.cpp`) returns `true` unconditionally, because every allocation it makes is `HOST_VISIBLE|HOST_COHERENT` and persistently mapped. Since [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477), which moved the portable reference tier off the WIDER `UnifiedMemory()`) `MetalBackend` (`src/vt/metal/metal_backend.mm`) answers `MetalContext::unified_memory()`, i.e. `dev.hasUnifiedMemory`, and `RocmBackend` (`src/vt/rocm/rocm_backend.hip`) answers its own `unified_memory_`, i.e. a managed allocator or an integrated part with `PageableMemoryAccess`. **So the lever ACTS on Apple silicon and on an integrated ROCm part, and nobody has measured it on either** — every number above is GB10 through Vulkan. Read those two as reachable-and-unmeasured rather than as covered by this row's numbers ([#1502](https://github.com/mudler/vllm.cpp/issues/1502)). It is still inert on CUDA and CPU, neither of which overrides the default: `tests/vllm/platforms/test_platform.cpp` pins `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())`, and the CPU backend reports `UnifiedMemory() == true` while this narrower predicate stays `false` — the two properties disagreeing is the whole reason the second one exists | +| `VT_ADOPT_DEVICE_BYTES` | on (acts only where the backend advertises host-addressable device memory: Vulkan always, Metal and ROCm where the part reports unified memory. MEASURED on Vulkan only) | After a weight is uploaded, re-point its host buffer AT the device allocation instead of keeping a second copy. On a unified box the two copies come out of the same RAM. MEASURED on GB10, Qwen3.6-27B bf16 (50.89 GiB): with the mirror the process reaches **VmRSS 100.759 GiB** and drives the machine to MemAvailable 13.85 / MemFree 1.13 GiB of 119.6 GiB before it has even finished allocating; without it the same load completes at **VmHWM 53.413 GiB**. Qwen3-4B: **16.392 -> 9.607 GiB**. Vulkan allocation is byte-identical either way. `0` is the same-binary A/B back to the two-copy behaviour, and it moves ONLY this lever: the `VT_LOAD_DIRECT_UPLOAD` post-upload source-page release still runs at `0` and on a backend without host-addressable device memory, so the two knobs stay independent. It is an adoption, not a release — the bytes survive at the device address and every reader sees them — so tokens are unchanged either way (`test_opt_paged_engine` on Vulkan is 6/6 token-exact, 96/96, both arms). **REACH AND MEASUREMENT ARE NOT THE SAME SET**, and this row used to read as if they were. The lever is gated on `vt::Backend::DeviceMemoryIsHostAddressable()`, read at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`. That predicate defaults to `false` (`include/vt/backend.h`), so a backend has to opt in, and THREE now do. `VulkanBackend` (`src/vt/vulkan/vulkan_backend.cpp`) returns `true` unconditionally, because every allocation it makes is `HOST_VISIBLE|HOST_COHERENT` and persistently mapped. Since [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477), which moved the portable reference tier off the WIDER `UnifiedMemory()`) `MetalBackend` (`src/vt/metal/metal_backend.mm`) answers `MetalContext::unified_memory()`, i.e. `dev.hasUnifiedMemory`, and `RocmBackend` (`src/vt/rocm/rocm_backend.hip`) answers its own `unified_memory_`, i.e. a managed allocator or an integrated part with `PageableMemoryAccess`. **So the lever ACTS on Apple silicon and on an integrated ROCm part, and nobody has measured it on either** — every number above is GB10 through Vulkan. Read those two as reachable-and-unmeasured rather than as covered by this row's numbers ([#1502](https://github.com/mudler/vllm.cpp/issues/1502)). It is still inert on CUDA and CPU, neither of which overrides the default — and that holds by ABSENCE of an override in `CudaBackend` (`include/vt/backend.h` defaults it `false`), NOT by a test. `tests/vllm/platforms/test_platform.cpp` looks like it pins the CUDA answer and does not: its `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` but its `backend()` returns `vt::GetBackend(DeviceType::kCPU)`, so the `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` at `:329` reads the CPU backend. Nothing in the tree pins the real `CudaBackend` ([#1635](https://github.com/mudler/vllm.cpp/issues/1635)). The CPU backend reports `UnifiedMemory() == true` while this narrower predicate stays `false` — the two properties disagreeing is the whole reason the second one exists | | `VT_QWEN35_ALIAS_HOST_WEIGHTS` | on (Qwen3.5/3.6 family; acts only where the platform advertises `host_memory_is_device_addressable()` — a GB10-class integrated part today) | Hand a dense weight's HOST bytes straight to the device kernel instead of allocating a device copy and uploading into it. On a part whose kernels can dereference host storage that copy buys nothing and costs a second full resident copy of every dense weight out of the same RAM. MEASURED on GB10, `Qwen3.8-2.4T-A95B UD-Q1_0` (369.97 GiB) on `--device cuda`: with the second copy the load completes at 61.20 GiB resident and then exhausts the 119.631 GiB box inside its FIRST forward, **zero decode steps over seven attempts** (issue #1299); with the alias the same run reaches **32/32 decode steps at peak RSS 97.75 GiB**. The instrument counts 60.793 GiB of dense weight aliased rather than duplicated (first-forward totals, at call 1361, where re-homing plateaus) against ~9.2 GiB of misaligned GGUF borrows that decline and still stage. `0` is the same-binary A/B back to the staging behaviour, and it is not only a bisect lever: `src/vllm/model_executor/models/laguna.cpp` records a MEASURED GB10 penalty for reading system-allocated memory from the GPU rather than a `cudaMalloc` allocation, worst on a long-K low-parallelism GEMV, so a decode regression has to be separable from the workload. The substitution is otherwise indistinguishable — the aliased pointer is re-homed to 256 bytes, cuBLASLt's own `CUBLASLT_MATMUL_PREF_MIN_ALIGNMENT_A_BYTES` default, which dominates every explicit pointer gate in the CUDA kernels (the strictest asks 32). A DISCRETE device answers the predicate false and is byte-for-byte unchanged. No effect on CPU, Vulkan, Metal or XPU, whose platforms do not advertise the property | | `VT_LOAD_DIRECT_UPLOAD` | on | Load a weight the device consumes VERBATIM by VIEWING the safetensors mmap (`OwnedBytes::Borrow`, keep-alive on the mapping) instead of copying it into an owned host buffer first, so the device upload reads the file mapping and the load moves those bytes ONCE rather than twice. Only whole-range same-size copies qualify — a transpose, a dtype conversion, a dequant, a concatenation or a load-time repack always takes the copy path, and the helper re-checks `numel * sizeof(dtype) == span` and fails closed to the copy on any mismatch. `0` is the same-binary A/B back to copy-then-upload. Bytes are identical either way, so tokens are unchanged. MEASURED on GB10, Qwen3.6-27B bf16 (50.098 GiB), Vulkan, same binary both arms: the weight-load phase goes **19.27 -> 12.48 s warm** (1.54x) and **52.62 -> 32.75 s cold** (1.61x), load-and-one-token **30.39 -> 22.47 s** warm and **62.98 -> 55.60 s** cold, with every ON leg beating every OFF leg. Total bytes MOVED **100.196 -> 81.260 GiB**: the host materialization pass drops **50.098 -> 31.162 GiB** while the 50.098 GiB device upload is unchanged (the model still has to be uploaded once). 37.8% of this checkpoint qualifies; the rest is merged (qkv, gate_up) or transposed (lm_head) at load and correctly still copies | | `VT_LOAD_STATS` | off | `=1` prints one line per load phase (mmap+header, weights) with its wall time, plus the bytes the load MOVED: `host_copy` (source bytes materialized into an owned host buffer), `borrowed` (source bytes viewed in place by the direct-upload path) and `device_upload` (bytes copied host to device). Diagnostic only; it changes no numerics. Issue #150 | From d0a59a98a969a80cb0824e2e5ff7481123b6de85 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Fri, 21 Aug 2026 21:16:07 +0000 Subject: [PATCH 2/4] record(VT-REFTIER-HOST-ADDRESSABLE): move the owed item under `## Owed`, and cite the index by ROW MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two review findings on the CUDA unified-memory citation correction. The `#1635` bullet was written into `## Scope`, under "Out of scope, and why", because the authoring script anchored on `s.index("## Owed")` and matched a PROSE MENTION of that string inside `## Scope` — the "Recorded under `## Owed`" cross-reference in the `#1435` bullet above it — rather than the heading. It is now the last bullet of the real `## Owed` section, anchored on a line that begins `## Owed`. The content was miscategorised as well as misplaced: "#1635 is OPEN and owned here, and pinning stays owed" states an obligation, not an exclusion from scope. The row also baked a line number into the append-only index. It cited the wrong claim's home as `.agents/issue-index.md:552`, and that was already false: `origin/main` landed five rows above `#1502` while this branch was open, so the merged tree puts it at `:557`. A permanent line number into a file nobody may edit would have been wrong on the day it shipped — which is the defect this row exists to describe, committed inside the row describing it, and it is `#844`'s own rule. Every reference now names the ROW: "the `#1502` row of this index", and "the `VT_ADOPT_DEVICE_BYTES` row of `docs/ENVIRONMENT.md`". The second was lower risk than the first, because `docs/ENVIRONMENT.md` is editable, but a line number is a line number and the row already had to say where the claim lives; naming it costs nothing and cannot rot. A corrected `:557` was rejected: it rots again on the next append above it. Editing the `#1635` row rather than appending another is not an append-only violation. The row exists only on this branch — `git show origin/main:.agents/issue-index.md` has no `#1635` row — and the rule forbids editing a row already in the log, which this is not. `issue-index append-only` passes. Records only. No product code, no test, and no behaviour changes. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 2 +- .../vt-reference-tier-host-addressable.md | 23 ++++++++++++------- 2 files changed, 16 insertions(+), 9 deletions(-) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 69ce0f05a..63f7ac1c3 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -550,7 +550,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1613](https://github.com/mudler/vllm.cpp/issues/1613) | `GATE-QWEN38-27B-FP8-BLOCK` | **The `Qwen/Qwen3.8-27B-FP8` block-wise token gate cannot be taken, because the 28.75 GiB checkpoint is not on the share.** `/mnt/nas_share/rc/ckpt/` holds `qwen3.8-27b-hf`, which is the **bf16** artifact -- no `quantization_config` key, `text_config.dtype = bfloat16` -- and `qwen3.8-q1_0`. Neither is this subject. The share has 3.4 TiB free, so the cost is AUTHORITY: `.agents/developer-preferences.md` authorizes large downloads for the `SPEC-DFLASH2` assets only. Nothing else blocks the gate, and that was not known before: a range-request audit of all 66 shard headers at revision `017b9c7a` shows every one of the 407 `F8_E4M3` tensors has `N % 128 == 0` and `K % 128 == 0`, so the sm120 complete-scale-block refusal (#1453) that makes DSV3's `kv_a_proj_with_mqa` unservable blocks NOTHING here; the ragged GDN `in_proj_a`/`in_proj_b` `[48, 5120]` are `BF16` and named in `modules_to_not_convert`; `weight_scale_inv` ships `BF16` (byte-checked via `data_offsets`, not the label) which `LoadFp8BlockRaw` already widens by value; and the per-layer `layers-.safetensors` naming already resolves through `SelectWeightFiles`. Spec `.agents/specs/gate-qwen38-27b-fp8-block.md`, parent #1189 | gap | | [#1614](https://github.com/mudler/vllm.cpp/issues/1614) | `GATE-QWEN38-27B-FP8-BLOCK` | **Three sites said `Qwen/Qwen3.8-27B-FP8` ships "~400" `modules_to_not_convert` entries, and at revision `017b9c7a` it ships 882** (882 unique, 636 outside the vision tower). The number is the evidence for an ARGUMENT -- it is why `IsFp8BlockProjection` reads the config AND the tensors instead of probing dtypes -- so being wrong by more than 2.2x invites the next reader to re-derive it. No reading of the list produces ~400: the visual entries are duplicated under two naming conventions, so distinct modules are about 759, and half of 882 is 441. Sites: the comment above `IsFp8BlockProjection`, the comment above `Fp8BlockQuantConfig::modules_to_not_convert`, and `.agents/specs/model-fp8-block-weight.md`. The routing itself is correct and no defect in it is asserted; two other claims in the same comment were checked against the checkpoint headers and hold (zero `input_scale` tensors, and the `[96, 40]` block-grid hazard is real). Found while auditing the checkpoint for #1613, fixed in the same flow | bug | | [#1502](https://github.com/mudler/vllm.cpp/issues/1502) | `VT-REFTIER-HOST-ADDRESSABLE` | **`docs/ENVIRONMENT.md` described `VT_ADOPT_DEVICE_BYTES` as Vulkan-only and said it has "No effect on CUDA/CPU/Metal", and [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477)) made both halves false.** That change moved `ReferenceTierEligible` off `UnifiedMemory()` onto `Backend::DeviceMemoryIsHostAddressable()` and added truthful overrides so no backend lost the reference tier, so `MetalBackend` now answers `MetalContext::unified_memory()` and `RocmBackend` answers its `unified_memory_`. The weight loader gates the lever on exactly that predicate, at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`, so the lever ACTS on Apple silicon and on an integrated ROCm part. **The correction is not "add two backend names".** Every number in that row is GB10 through Vulkan, and nobody has measured the lever on either new arm, so the row now separates the backends it is MEASURED on from the backends that merely satisfy the predicate — reach and measurement are different claims and the row read as if the measurement covered the reach. CUDA and CPU stay inert and are unchanged: neither overrides the default `false`, which `tests/vllm/platforms/test_platform.cpp` pins for GB10, and the CPU backend answering `UnifiedMemory() == true` while the narrower predicate stays `false` is the whole reason the two properties are separate. The MEASUREMENT on Metal and integrated ROCm stays owed and is listed under `## Owed` in [`vt-reference-tier-host-addressable.md`](specs/vt-reference-tier-host-addressable.md); it needs an Apple-silicon box or an integrated AMD part | documentation | -| [#1635](https://github.com/mudler/vllm.cpp/issues/1635) | `VT-REFTIER-HOST-ADDRESSABLE` | **[#1620](https://github.com/mudler/vllm.cpp/pull/1620) cited `tests/vllm/platforms/test_platform.cpp` as pinning the CUDA backend's `DeviceMemoryIsHostAddressable()` default, and that test reads the CPU backend.** `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` (`:284`) while its `backend()` returns `vt::GetBackend(DeviceType::kCPU)` (`:285`), so the `CHECK_FALSE` at `:329` asserts against CPU and the test's own comment says so. **Nothing in the tree pins the real `CudaBackend`.** The CONCLUSION survives — `CudaBackend` declares no override, so it inherits the base `false` at `include/vt/backend.h:77` — but it holds by absence, which is a weaker claim than a pin and must not read as one. No gate could catch this: the assertion passes, and it passes for a reason unrelated to what it claims to measure. The wrong citation landed in three places and one is PERMANENT: `docs/ENVIRONMENT.md:210` (corrected in this flow), the commit message, and `.agents/issue-index.md:552`, which is append-only and can never be edited — which is why this correction needs a row of its own rather than an edit. Found by post-hoc review of four merged records pull requests, not by a gate. OWED: either pin the real `CudaBackend` or state in the record that the default holds unpinned | bug | +| [#1635](https://github.com/mudler/vllm.cpp/issues/1635) | `VT-REFTIER-HOST-ADDRESSABLE` | **[#1620](https://github.com/mudler/vllm.cpp/pull/1620) cited `tests/vllm/platforms/test_platform.cpp` as pinning the CUDA backend's `DeviceMemoryIsHostAddressable()` default, and that test reads the CPU backend.** `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` (`:284`) while its `backend()` returns `vt::GetBackend(DeviceType::kCPU)` (`:285`), so the `CHECK_FALSE` at `:329` asserts against CPU and the test's own comment says so. **Nothing in the tree pins the real `CudaBackend`.** The CONCLUSION survives — `CudaBackend` declares no override, so it inherits the base `false` at `include/vt/backend.h:77` — but it holds by absence, which is a weaker claim than a pin and must not read as one. No gate could catch this: the assertion passes, and it passes for a reason unrelated to what it claims to measure. The wrong citation landed in three places and one is PERMANENT: the `VT_ADOPT_DEVICE_BYTES` row of `docs/ENVIRONMENT.md` (corrected in this flow), the commit message, and the `#1502` row of this index, which is append-only and can never be edited — which is why this correction needs a row of its own rather than an edit. Both are named by ROW and not by line on purpose: the first draft of this row cited the `#1502` row by LINE NUMBER, and merging the `origin/main` that had landed under it in the meantime pushed that row five lines further down before this row had shipped even once, so a permanent line number into an append-only file would have been wrong on the day it landed — which is [#844](https://github.com/mudler/vllm.cpp/issues/844)'s rule, committed inside the row that exists to describe this class of defect. Found by post-hoc review of four merged records pull requests, not by a gate. OWED: either pin the real `CudaBackend` or state in the record that the default holds unpinned | bug | | [#1538](https://github.com/mudler/vllm.cpp/issues/1538) | `SPEC-DFLASH2` | **vllm#52816's head moved a THIRD time (`66e5414c` -> `3406ec1d`) while it is still open, and refactors `compute_candidates` into `LogitsProcessor.get_top_k_tokens`.** Measured 2026-08-21 by W6 from `raw.githubusercontent.com` at both heads: +11/-80 on `qwen3_dflash2.py`, +4/-16 on `dflash2/speculator.py`, +2/-5 on the base `speculator.py`. The big one is a RELOCATION rather than new math -- the padding mask, the id rebase, the TP all-gather and the scale-THEN-softcap order all survive in `logits_processor.py:241-286`, so `## Owed` O16's reading of the codebook-span question holds at BOTH heads. What does NOT survive is the explicit `UnquantizedEmbeddingMethod`/`UnquantizedLinearMethod` guard that `## Risks/decisions` D12 ports as `RefuseQuantizedDflash2LmHead`, which is deleted at `3406ec1d`; our guard's own reason (the GGUF arm dequantizes `output.weight` to bf16, and a GGUF target with a safetensors DFlash2 draft is admitted here) is independent of upstream's and stands. NOT reconciled in flow, deliberately: `## Gates` G2 fixes the gate head at `66e5414c` while the pull request is unmerged, and moving the port onto a third unmerged head during the gate would move the thing being measured. Owed under `## Owed` O21 of [the DFlash2 spec](specs/dflash2-spec-decode.md) | verification | | [#1456](https://github.com/mudler/vllm.cpp/issues/1456) | `SPEC-DFLASH2` | **The GB10 oracle DOES have a FLASH_ATTN denominator: the arch measurement stands, the conclusion drawn from it does not.** Measured 2026-08-21 by W6 on `dgx:gpu0` through an `rc` lease, with the very wheel #1456 was filed about (`vllm-0.1.dev1+g66e5414c6`, sha256 `fbc247ab...`). A capture that exported `VLLM_ATTENTION_BACKEND=TRITON_ATTN` got `FLASH_ATTN` anyway and RAN: `Using FlashAttention version 2`, 54.87 GiB loaded, CUDA graphs captured (PIECEWISE 5/5, FULL 1/1, plus the DFlash2 speculator's own), 4 x 64 coherent tokens, speculation live at 209 accepted of 350 drafted and mean acceptance length 5.00. No `cudaErrorUnsupportedPtxVersion`. Consistent with the `sm_80`/`sm_75` SASS finding rather than contradicting it: `sm_80` PTX JITs FORWARD, and that error is the OPPOSITE failure (PTX newer than the driver). So `FA_USABLE=0` in the staged `FA-CONSTRAINT.txt` was inferred from emitted arches, never observed from a run, and is the thing to reconcile. A SECOND trap found in the same run and recorded so nobody repeats it: **`VLLM_ATTENTION_BACKEND` does not exist at this revision** -- grepping every `.py` in the wheel returns nothing; the knob is `EngineArgs.attention_backend` (`arg_utils.py:706`) folded into `AttentionConfig.backend` (`:2382`), so the old export selects NOTHING and auto-selection wins silently, letting a run record one backend while executing another. NOT reconciled in flow: W6 does not substitute a denominator the developer declared, and takes both arms instead, each named in its own golden. Owed under `## Owed` O22 of [the DFlash2 spec](specs/dflash2-spec-decode.md) | verification | | [#1561](https://github.com/mudler/vllm.cpp/issues/1561) | `SPEC-DFLASH2` | **vllm#52816 MERGED, so this row's gate head and O21's parked D12 decision both come due.** Read from the forge 2026-08-21: `merged: true`, `merged_at 2026-08-21T05:27:22Z`, merge commit `b389ac29465b33f9e9c534df221ea3c129e9793f`, head `3406ec1dae9916f920b90f0dbf90dcf54923d042`. The merge landed 46 minutes BEFORE `SPEC-DFLASH2` W6's work commit `bb416e0ae` was authored (`06:13:50Z`), so five statements in the spec plus the `#1538` index row recorded an open pull request that had already closed. `## Gates` G2's own rule is "`66e5414c` if #52816 has not merged, and the merge commit if it has", so the head it selects today is `b389ac29`. W6's capture stays pinned to `66e5414c` because that is the wheel that ran and it predates the merge -- a dated exception, not the rule -- and what is owed here is moving the gate head and re-reading G2 and G3 at the merged head. O21 additionally parked the `UnquantizedEmbeddingMethod`/`UnquantizedLinearMethod` guard deletion as "the decision when #52816 settles"; it has settled onto vLLM's `main`, our `RefuseQuantizedDflash2LmHead` stands on its own independent reason (the GGUF arm dequantizes `output.weight` to bf16), and writing that decision down against a merged upstream rather than a branch is owed. Re-verified at `3406ec1d`: `logits_processor.py:241-286` preserves the padding mask, the id rebase, the TP all-gather and the scale-THEN-softcap order, so O16's reading holds at the merged head too. Records corrected on `row/SPEC-DFLASH2-W6`; the WORK is owed here | verification | diff --git a/.agents/specs/vt-reference-tier-host-addressable.md b/.agents/specs/vt-reference-tier-host-addressable.md index 4a4a997fb..c79749499 100644 --- a/.agents/specs/vt-reference-tier-host-addressable.md +++ b/.agents/specs/vt-reference-tier-host-addressable.md @@ -79,14 +79,6 @@ Out of scope, and why: authoring host has no `nvcc` and no GPU. Bundling it here would put a change that only a leased box can verify inside a change that a CPU box verifies completely. Recorded under [`## Owed`](#owed). - -- [#1635](https://github.com/mudler/vllm.cpp/issues/1635) is OPEN and owned here. The record's - claim that `tests/vllm/platforms/test_platform.cpp` pins the CUDA - `DeviceMemoryIsHostAddressable()` default is FALSE: that fixture's `backend()` returns the CPU - backend, so the assertion never reads `CudaBackend` at all. The conclusion survives by absence of - an override; the pin does not exist. `docs/ENVIRONMENT.md` is corrected, but the copy in - `.agents/issue-index.md:552` is append-only and permanent. Pinning the real `CudaBackend` stays - owed and needs a CUDA device to exercise. - **Copy-in and copy-out instead of refusal.** Rejected under [Design](#design). - **Registering a CUDA kernel for `kMatmulFp8BlockScaled` on a CUTLASS-less @@ -304,6 +296,21 @@ is the landed commit message. The two GB10 logs that motivated the row are the backends that merely satisfy the predicate. What stays owed is the MEASUREMENT itself on the two new arms, which needs an Apple-silicon box or an integrated AMD part and cannot be taken here. Owned by this row. +- **Nothing in the tree pins the real `CudaBackend`'s + `DeviceMemoryIsHostAddressable()`, and + [#1635](https://github.com/mudler/vllm.cpp/issues/1635) is OPEN and owned + here.** `tests/vllm/platforms/test_platform.cpp` was cited as that pin and is + not one: `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` + while its `backend()` returns `vt::GetBackend(DeviceType::kCPU)`, so the + `CHECK_FALSE` reads the CPU backend and the fixture's own comment says so. The + conclusion survives by ABSENCE of an override — `CudaBackend` declares none, so + it inherits the base `false` in `include/vt/backend.h` — which is a weaker + claim than a pin and must not read as one. `docs/ENVIRONMENT.md` is corrected + here; the `#1502` row in `.agents/issue-index.md` keeps the wrong citation, + because that index is append-only and can never be edited. What stays owed is + the pin itself: either exercise the real `CudaBackend`, which needs a CUDA + device, or state in the record that the default holds unpinned. Owned by this + row. ## Now From 310040faf52074a169fc9107e1f3797439a28a3e Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Fri, 21 Aug 2026 21:49:24 +0000 Subject: [PATCH 3/4] record(VT-REFTIER-HOST-ADDRESSABLE): cite the fixture by SYMBOL, because #844's rule is about the file the number is written IN The `#1635` row invoked #844's rule -- "grep the symbol, because this index is append-only and a line number here would be permanent" -- while itself keeping four permanent line numbers: `:284`, `:285` and `:329` into `tests/vllm/platforms/test_platform.cpp`, and `include/vt/backend.h:77`. The distinction the previous wording drew was "editable target, so keep the anchor". That is not #844's distinction. #844's criterion is the file the number is WRITTEN IN, not the file it POINTS AT: this index is append-only and no checker scans it, so a number here can never be repaired once it rots, no matter how editable its target is. A row that argues a rule and then breaks it teaches the wrong criterion to the next reader. The four numbers are replaced by names that cannot rot: the `device_type()` and `backend()` overrides of `FakeUnifiedAddressablePlatform`, the assertion `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())`, and `Backend::DeviceMemoryIsHostAddressable`'s `false` default. The file names stay, so the row is still grep-navigable. The row also now states that the landing pull request carries no closing keyword and that #1635 stays OPEN. It carried `Fixes #1635` in its body, which would have closed the issue at the squash and falsified this row's own OWED clause and the `## Owed` bullet in `.agents/specs/vt-reference-tier-host-addressable.md`, both of which say #1635 is open. The second owed item -- pinning the real `CudaBackend`, which needs a CUDA device -- is not done here, so closing it would leave that work tracked by a spec bullet alone. Only the `#1635` row is touched. It exists on this branch only and is not yet in the log, so this is not an append-only violation: `check-issue-index-append-only.py` diffs `merge-base(origin/main, HEAD)..HEAD` and reports ok. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 87ae514e5..d4535da52 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -555,7 +555,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1613](https://github.com/mudler/vllm.cpp/issues/1613) | `GATE-QWEN38-27B-FP8-BLOCK` | **The `Qwen/Qwen3.8-27B-FP8` block-wise token gate cannot be taken, because the 28.75 GiB checkpoint is not on the share.** `/mnt/nas_share/rc/ckpt/` holds `qwen3.8-27b-hf`, which is the **bf16** artifact -- no `quantization_config` key, `text_config.dtype = bfloat16` -- and `qwen3.8-q1_0`. Neither is this subject. The share has 3.4 TiB free, so the cost is AUTHORITY: `.agents/developer-preferences.md` authorizes large downloads for the `SPEC-DFLASH2` assets only. Nothing else blocks the gate, and that was not known before: a range-request audit of all 66 shard headers at revision `017b9c7a` shows every one of the 407 `F8_E4M3` tensors has `N % 128 == 0` and `K % 128 == 0`, so the sm120 complete-scale-block refusal (#1453) that makes DSV3's `kv_a_proj_with_mqa` unservable blocks NOTHING here; the ragged GDN `in_proj_a`/`in_proj_b` `[48, 5120]` are `BF16` and named in `modules_to_not_convert`; `weight_scale_inv` ships `BF16` (byte-checked via `data_offsets`, not the label) which `LoadFp8BlockRaw` already widens by value; and the per-layer `layers-.safetensors` naming already resolves through `SelectWeightFiles`. Spec `.agents/specs/gate-qwen38-27b-fp8-block.md`, parent #1189 | gap | | [#1614](https://github.com/mudler/vllm.cpp/issues/1614) | `GATE-QWEN38-27B-FP8-BLOCK` | **Three sites said `Qwen/Qwen3.8-27B-FP8` ships "~400" `modules_to_not_convert` entries, and at revision `017b9c7a` it ships 882** (882 unique, 636 outside the vision tower). The number is the evidence for an ARGUMENT -- it is why `IsFp8BlockProjection` reads the config AND the tensors instead of probing dtypes -- so being wrong by more than 2.2x invites the next reader to re-derive it. No reading of the list produces ~400: the visual entries are duplicated under two naming conventions, so distinct modules are about 759, and half of 882 is 441. Sites: the comment above `IsFp8BlockProjection`, the comment above `Fp8BlockQuantConfig::modules_to_not_convert`, and `.agents/specs/model-fp8-block-weight.md`. The routing itself is correct and no defect in it is asserted; two other claims in the same comment were checked against the checkpoint headers and hold (zero `input_scale` tensors, and the `[96, 40]` block-grid hazard is real). Found while auditing the checkpoint for #1613, fixed in the same flow | bug | | [#1502](https://github.com/mudler/vllm.cpp/issues/1502) | `VT-REFTIER-HOST-ADDRESSABLE` | **`docs/ENVIRONMENT.md` described `VT_ADOPT_DEVICE_BYTES` as Vulkan-only and said it has "No effect on CUDA/CPU/Metal", and [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477)) made both halves false.** That change moved `ReferenceTierEligible` off `UnifiedMemory()` onto `Backend::DeviceMemoryIsHostAddressable()` and added truthful overrides so no backend lost the reference tier, so `MetalBackend` now answers `MetalContext::unified_memory()` and `RocmBackend` answers its `unified_memory_`. The weight loader gates the lever on exactly that predicate, at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`, so the lever ACTS on Apple silicon and on an integrated ROCm part. **The correction is not "add two backend names".** Every number in that row is GB10 through Vulkan, and nobody has measured the lever on either new arm, so the row now separates the backends it is MEASURED on from the backends that merely satisfy the predicate — reach and measurement are different claims and the row read as if the measurement covered the reach. CUDA and CPU stay inert and are unchanged: neither overrides the default `false`, which `tests/vllm/platforms/test_platform.cpp` pins for GB10, and the CPU backend answering `UnifiedMemory() == true` while the narrower predicate stays `false` is the whole reason the two properties are separate. The MEASUREMENT on Metal and integrated ROCm stays owed and is listed under `## Owed` in [`vt-reference-tier-host-addressable.md`](specs/vt-reference-tier-host-addressable.md); it needs an Apple-silicon box or an integrated AMD part | documentation | -| [#1635](https://github.com/mudler/vllm.cpp/issues/1635) | `VT-REFTIER-HOST-ADDRESSABLE` | **[#1620](https://github.com/mudler/vllm.cpp/pull/1620) cited `tests/vllm/platforms/test_platform.cpp` as pinning the CUDA backend's `DeviceMemoryIsHostAddressable()` default, and that test reads the CPU backend.** `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` (`:284`) while its `backend()` returns `vt::GetBackend(DeviceType::kCPU)` (`:285`), so the `CHECK_FALSE` at `:329` asserts against CPU and the test's own comment says so. **Nothing in the tree pins the real `CudaBackend`.** The CONCLUSION survives — `CudaBackend` declares no override, so it inherits the base `false` at `include/vt/backend.h:77` — but it holds by absence, which is a weaker claim than a pin and must not read as one. No gate could catch this: the assertion passes, and it passes for a reason unrelated to what it claims to measure. The wrong citation landed in three places and one is PERMANENT: the `VT_ADOPT_DEVICE_BYTES` row of `docs/ENVIRONMENT.md` (corrected in this flow), the commit message, and the `#1502` row of this index, which is append-only and can never be edited — which is why this correction needs a row of its own rather than an edit. Both are named by ROW and not by line on purpose: the first draft of this row cited the `#1502` row by LINE NUMBER, and merging the `origin/main` that had landed under it in the meantime pushed that row five lines further down before this row had shipped even once, so a permanent line number into an append-only file would have been wrong on the day it landed — which is [#844](https://github.com/mudler/vllm.cpp/issues/844)'s rule, committed inside the row that exists to describe this class of defect. Found by post-hoc review of four merged records pull requests, not by a gate. OWED: either pin the real `CudaBackend` or state in the record that the default holds unpinned | bug | +| [#1635](https://github.com/mudler/vllm.cpp/issues/1635) | `VT-REFTIER-HOST-ADDRESSABLE` | **[#1620](https://github.com/mudler/vllm.cpp/pull/1620) cited `tests/vllm/platforms/test_platform.cpp` as pinning the CUDA backend's `DeviceMemoryIsHostAddressable()` default, and that test reads the CPU backend.** `FakeUnifiedAddressablePlatform`'s `device_type()` override returns `DeviceType::kCUDA` while its `backend()` override returns `vt::GetBackend(DeviceType::kCPU)`, so the assertion `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` reads CPU and the test's own comment says so. Grep those three names; do not look for a line number, because this index is append-only and a number written HERE is permanent whatever file it points at. **Nothing in the tree pins the real `CudaBackend`.** The CONCLUSION survives — `CudaBackend` declares no override, so it inherits `Backend::DeviceMemoryIsHostAddressable`'s `false` default in `include/vt/backend.h` — but it holds by absence, which is a weaker claim than a pin and must not read as one. No gate could catch this: the assertion passes, and it passes for a reason unrelated to what it claims to measure. The wrong citation landed in three places and one is PERMANENT: the `VT_ADOPT_DEVICE_BYTES` row of `docs/ENVIRONMENT.md` (corrected in this flow), the commit message, and the `#1502` row of this index, which is append-only and can never be edited — which is why this correction needs a row of its own rather than an edit. Both are named by ROW and not by line on purpose: the first draft of this row cited the `#1502` row by LINE NUMBER, and merging the `origin/main` that had landed under it in the meantime pushed that row five lines further down before this row had shipped even once, so a permanent line number into an append-only file would have been wrong on the day it landed — which is [#844](https://github.com/mudler/vllm.cpp/issues/844)'s rule, committed inside the row that exists to describe this class of defect. Found by post-hoc review of four merged records pull requests, not by a gate. The pull request that lands this row carries NO closing keyword and #1635 stays OPEN: this change corrects the citation and records the gap, while the pin itself needs a CUDA device. OWED: either pin the real `CudaBackend` or state in the record that the default holds unpinned | bug | | [#1538](https://github.com/mudler/vllm.cpp/issues/1538) | `SPEC-DFLASH2` | **vllm#52816's head moved a THIRD time (`66e5414c` -> `3406ec1d`) while it is still open, and refactors `compute_candidates` into `LogitsProcessor.get_top_k_tokens`.** Measured 2026-08-21 by W6 from `raw.githubusercontent.com` at both heads: +11/-80 on `qwen3_dflash2.py`, +4/-16 on `dflash2/speculator.py`, +2/-5 on the base `speculator.py`. The big one is a RELOCATION rather than new math -- the padding mask, the id rebase, the TP all-gather and the scale-THEN-softcap order all survive in `logits_processor.py:241-286`, so `## Owed` O16's reading of the codebook-span question holds at BOTH heads. What does NOT survive is the explicit `UnquantizedEmbeddingMethod`/`UnquantizedLinearMethod` guard that `## Risks/decisions` D12 ports as `RefuseQuantizedDflash2LmHead`, which is deleted at `3406ec1d`; our guard's own reason (the GGUF arm dequantizes `output.weight` to bf16, and a GGUF target with a safetensors DFlash2 draft is admitted here) is independent of upstream's and stands. NOT reconciled in flow, deliberately: `## Gates` G2 fixes the gate head at `66e5414c` while the pull request is unmerged, and moving the port onto a third unmerged head during the gate would move the thing being measured. Owed under `## Owed` O21 of [the DFlash2 spec](specs/dflash2-spec-decode.md) | verification | | [#1456](https://github.com/mudler/vllm.cpp/issues/1456) | `SPEC-DFLASH2` | **The GB10 oracle DOES have a FLASH_ATTN denominator: the arch measurement stands, the conclusion drawn from it does not.** Measured 2026-08-21 by W6 on `dgx:gpu0` through an `rc` lease, with the very wheel #1456 was filed about (`vllm-0.1.dev1+g66e5414c6`, sha256 `fbc247ab...`). A capture that exported `VLLM_ATTENTION_BACKEND=TRITON_ATTN` got `FLASH_ATTN` anyway and RAN: `Using FlashAttention version 2`, 54.87 GiB loaded, CUDA graphs captured (PIECEWISE 5/5, FULL 1/1, plus the DFlash2 speculator's own), 4 x 64 coherent tokens, speculation live at 209 accepted of 350 drafted and mean acceptance length 5.00. No `cudaErrorUnsupportedPtxVersion`. Consistent with the `sm_80`/`sm_75` SASS finding rather than contradicting it: `sm_80` PTX JITs FORWARD, and that error is the OPPOSITE failure (PTX newer than the driver). So `FA_USABLE=0` in the staged `FA-CONSTRAINT.txt` was inferred from emitted arches, never observed from a run, and is the thing to reconcile. A SECOND trap found in the same run and recorded so nobody repeats it: **`VLLM_ATTENTION_BACKEND` does not exist at this revision** -- grepping every `.py` in the wheel returns nothing; the knob is `EngineArgs.attention_backend` (`arg_utils.py:706`) folded into `AttentionConfig.backend` (`:2382`), so the old export selects NOTHING and auto-selection wins silently, letting a run record one backend while executing another. NOT reconciled in flow: W6 does not substitute a denominator the developer declared, and takes both arms instead, each named in its own golden. Owed under `## Owed` O22 of [the DFlash2 spec](specs/dflash2-spec-decode.md) | verification | | [#1561](https://github.com/mudler/vllm.cpp/issues/1561) | `SPEC-DFLASH2` | **vllm#52816 MERGED, so this row's gate head and O21's parked D12 decision both come due.** Read from the forge 2026-08-21: `merged: true`, `merged_at 2026-08-21T05:27:22Z`, merge commit `b389ac29465b33f9e9c534df221ea3c129e9793f`, head `3406ec1dae9916f920b90f0dbf90dcf54923d042`. The merge landed 46 minutes BEFORE `SPEC-DFLASH2` W6's work commit `bb416e0ae` was authored (`06:13:50Z`), so five statements in the spec plus the `#1538` index row recorded an open pull request that had already closed. `## Gates` G2's own rule is "`66e5414c` if #52816 has not merged, and the merge commit if it has", so the head it selects today is `b389ac29`. W6's capture stays pinned to `66e5414c` because that is the wheel that ran and it predates the merge -- a dated exception, not the rule -- and what is owed here is moving the gate head and re-reading G2 and G3 at the merged head. O21 additionally parked the `UnquantizedEmbeddingMethod`/`UnquantizedLinearMethod` guard deletion as "the decision when #52816 settles"; it has settled onto vLLM's `main`, our `RefuseQuantizedDflash2LmHead` stands on its own independent reason (the GGUF arm dequantizes `output.weight` to bf16), and writing that decision down against a merged upstream rather than a branch is owed. Re-verified at `3406ec1d`: `logits_processor.py:241-286` preserves the padding mask, the id rebase, the TP all-gather and the scale-THEN-softcap order, so O16's reading holds at the merged head too. Records corrected on `row/SPEC-DFLASH2-W6`; the WORK is owed here | verification | From 34a8ca3a441728bd5f724aebd270cbe9b9b38fe6 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Fri, 21 Aug 2026 22:25:11 +0000 Subject: [PATCH 4/4] record(VT-REFTIER-HOST-ADDRESSABLE): the paragraph condemning a rotting line number carried a rotting count The `#1635` row and the pull request body both explained the line-number citation by saying `origin/main` had pushed the `#1502` row five lines further down. That was true at the previous merge and false at this head, where the figure is larger, because a third merge landed more rows above it. A count read off whatever `origin/main` happened to be merged last rots by exactly the mechanism the paragraph exists to condemn, and this branch will merge `origin/main` at least once more before it lands, so replacing five with the current figure would only schedule the next instance. Both sentences now carry no count and no line number. What survives is the argument, which no merge can falsify: `origin/main` appended rows above `#1502` while this branch was open, so the line the first draft named stopped identifying that row, which is why every citation names the row instead. The row also says outright that no count is written, so the next reader does not supply one. The `docs/ENVIRONMENT.md` sentence loses its `:329`. That file is editable, so a stale number there is repairable rather than permanent, but the sentence already quotes the assertion verbatim and the number adds nothing except one more thing to go stale -- in the very file this change is rewriting to remove a bad citation. Records only. No `src/`, `include/` or `tests/` path is touched, and #1635 stays open. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 2 +- docs/ENVIRONMENT.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 6d089d00e..c734e3562 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -557,7 +557,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1613](https://github.com/mudler/vllm.cpp/issues/1613) | `GATE-QWEN38-27B-FP8-BLOCK` | **The `Qwen/Qwen3.8-27B-FP8` block-wise token gate cannot be taken, because the 28.75 GiB checkpoint is not on the share.** `/mnt/nas_share/rc/ckpt/` holds `qwen3.8-27b-hf`, which is the **bf16** artifact -- no `quantization_config` key, `text_config.dtype = bfloat16` -- and `qwen3.8-q1_0`. Neither is this subject. The share has 3.4 TiB free, so the cost is AUTHORITY: `.agents/developer-preferences.md` authorizes large downloads for the `SPEC-DFLASH2` assets only. Nothing else blocks the gate, and that was not known before: a range-request audit of all 66 shard headers at revision `017b9c7a` shows every one of the 407 `F8_E4M3` tensors has `N % 128 == 0` and `K % 128 == 0`, so the sm120 complete-scale-block refusal (#1453) that makes DSV3's `kv_a_proj_with_mqa` unservable blocks NOTHING here; the ragged GDN `in_proj_a`/`in_proj_b` `[48, 5120]` are `BF16` and named in `modules_to_not_convert`; `weight_scale_inv` ships `BF16` (byte-checked via `data_offsets`, not the label) which `LoadFp8BlockRaw` already widens by value; and the per-layer `layers-.safetensors` naming already resolves through `SelectWeightFiles`. Spec `.agents/specs/gate-qwen38-27b-fp8-block.md`, parent #1189 | gap | | [#1614](https://github.com/mudler/vllm.cpp/issues/1614) | `GATE-QWEN38-27B-FP8-BLOCK` | **Three sites said `Qwen/Qwen3.8-27B-FP8` ships "~400" `modules_to_not_convert` entries, and at revision `017b9c7a` it ships 882** (882 unique, 636 outside the vision tower). The number is the evidence for an ARGUMENT -- it is why `IsFp8BlockProjection` reads the config AND the tensors instead of probing dtypes -- so being wrong by more than 2.2x invites the next reader to re-derive it. No reading of the list produces ~400: the visual entries are duplicated under two naming conventions, so distinct modules are about 759, and half of 882 is 441. Sites: the comment above `IsFp8BlockProjection`, the comment above `Fp8BlockQuantConfig::modules_to_not_convert`, and `.agents/specs/model-fp8-block-weight.md`. The routing itself is correct and no defect in it is asserted; two other claims in the same comment were checked against the checkpoint headers and hold (zero `input_scale` tensors, and the `[96, 40]` block-grid hazard is real). Found while auditing the checkpoint for #1613, fixed in the same flow | bug | | [#1502](https://github.com/mudler/vllm.cpp/issues/1502) | `VT-REFTIER-HOST-ADDRESSABLE` | **`docs/ENVIRONMENT.md` described `VT_ADOPT_DEVICE_BYTES` as Vulkan-only and said it has "No effect on CUDA/CPU/Metal", and [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477)) made both halves false.** That change moved `ReferenceTierEligible` off `UnifiedMemory()` onto `Backend::DeviceMemoryIsHostAddressable()` and added truthful overrides so no backend lost the reference tier, so `MetalBackend` now answers `MetalContext::unified_memory()` and `RocmBackend` answers its `unified_memory_`. The weight loader gates the lever on exactly that predicate, at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`, so the lever ACTS on Apple silicon and on an integrated ROCm part. **The correction is not "add two backend names".** Every number in that row is GB10 through Vulkan, and nobody has measured the lever on either new arm, so the row now separates the backends it is MEASURED on from the backends that merely satisfy the predicate — reach and measurement are different claims and the row read as if the measurement covered the reach. CUDA and CPU stay inert and are unchanged: neither overrides the default `false`, which `tests/vllm/platforms/test_platform.cpp` pins for GB10, and the CPU backend answering `UnifiedMemory() == true` while the narrower predicate stays `false` is the whole reason the two properties are separate. The MEASUREMENT on Metal and integrated ROCm stays owed and is listed under `## Owed` in [`vt-reference-tier-host-addressable.md`](specs/vt-reference-tier-host-addressable.md); it needs an Apple-silicon box or an integrated AMD part | documentation | -| [#1635](https://github.com/mudler/vllm.cpp/issues/1635) | `VT-REFTIER-HOST-ADDRESSABLE` | **[#1620](https://github.com/mudler/vllm.cpp/pull/1620) cited `tests/vllm/platforms/test_platform.cpp` as pinning the CUDA backend's `DeviceMemoryIsHostAddressable()` default, and that test reads the CPU backend.** `FakeUnifiedAddressablePlatform`'s `device_type()` override returns `DeviceType::kCUDA` while its `backend()` override returns `vt::GetBackend(DeviceType::kCPU)`, so the assertion `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` reads CPU and the test's own comment says so. Grep those three names; do not look for a line number, because this index is append-only and a number written HERE is permanent whatever file it points at. **Nothing in the tree pins the real `CudaBackend`.** The CONCLUSION survives — `CudaBackend` declares no override, so it inherits `Backend::DeviceMemoryIsHostAddressable`'s `false` default in `include/vt/backend.h` — but it holds by absence, which is a weaker claim than a pin and must not read as one. No gate could catch this: the assertion passes, and it passes for a reason unrelated to what it claims to measure. The wrong citation landed in three places and one is PERMANENT: the `VT_ADOPT_DEVICE_BYTES` row of `docs/ENVIRONMENT.md` (corrected in this flow), the commit message, and the `#1502` row of this index, which is append-only and can never be edited — which is why this correction needs a row of its own rather than an edit. Both are named by ROW and not by line on purpose: the first draft of this row cited the `#1502` row by LINE NUMBER, and merging the `origin/main` that had landed under it in the meantime pushed that row five lines further down before this row had shipped even once, so a permanent line number into an append-only file would have been wrong on the day it landed — which is [#844](https://github.com/mudler/vllm.cpp/issues/844)'s rule, committed inside the row that exists to describe this class of defect. Found by post-hoc review of four merged records pull requests, not by a gate. The pull request that lands this row carries NO closing keyword and #1635 stays OPEN: this change corrects the citation and records the gap, while the pin itself needs a CUDA device. OWED: either pin the real `CudaBackend` or state in the record that the default holds unpinned | bug | +| [#1635](https://github.com/mudler/vllm.cpp/issues/1635) | `VT-REFTIER-HOST-ADDRESSABLE` | **[#1620](https://github.com/mudler/vllm.cpp/pull/1620) cited `tests/vllm/platforms/test_platform.cpp` as pinning the CUDA backend's `DeviceMemoryIsHostAddressable()` default, and that test reads the CPU backend.** `FakeUnifiedAddressablePlatform`'s `device_type()` override returns `DeviceType::kCUDA` while its `backend()` override returns `vt::GetBackend(DeviceType::kCPU)`, so the assertion `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` reads CPU and the test's own comment says so. Grep those three names; do not look for a line number, because this index is append-only and a number written HERE is permanent whatever file it points at. **Nothing in the tree pins the real `CudaBackend`.** The CONCLUSION survives — `CudaBackend` declares no override, so it inherits `Backend::DeviceMemoryIsHostAddressable`'s `false` default in `include/vt/backend.h` — but it holds by absence, which is a weaker claim than a pin and must not read as one. No gate could catch this: the assertion passes, and it passes for a reason unrelated to what it claims to measure. The wrong citation landed in three places and one is PERMANENT: the `VT_ADOPT_DEVICE_BYTES` row of `docs/ENVIRONMENT.md` (corrected in this flow), the commit message, and the `#1502` row of this index, which is append-only and can never be edited — which is why this correction needs a row of its own rather than an edit. Both are named by ROW and not by line on purpose: the first draft of this row cited the `#1502` row by LINE NUMBER, and `origin/main` appended rows above it while this branch was open, so the line that draft named had stopped identifying the `#1502` row before this row had shipped even once, and a permanent line number into an append-only file would have been wrong on the day it landed — which is [#844](https://github.com/mudler/vllm.cpp/issues/844)'s rule, committed inside the row that exists to describe this class of defect. No COUNT of the rows that moved it is written here either: that number is read off whatever `origin/main` happened to be merged last, so it rots by the very mechanism this row condemns, and a draft of this row carried one that was already stale by the next merge. Found by post-hoc review of four merged records pull requests, not by a gate. The pull request that lands this row carries NO closing keyword and #1635 stays OPEN: this change corrects the citation and records the gap, while the pin itself needs a CUDA device. OWED: either pin the real `CudaBackend` or state in the record that the default holds unpinned | bug | | [#1632](https://github.com/mudler/vllm.cpp/issues/1632) | `QUANT-QWEN38-27B-NVFP4-ARM` | **W6's NVFP4 token gate named [#1185](https://github.com/mudler/vllm.cpp/issues/1185) as the authority it waits on, and #1185 closed on 2026-08-18 as local-only** -- it tracked one operator's machines rather than a defect here -- so five sites pointed a reader at an issue that reports "closed" without reporting "cleared": `docs/FEATURES.md`, and the spec's `**Related:**` header, wave table, blockers section, `## Owed` list and `## Now`. **The blocker did not close with the issue, and it is not the one the citations described.** The pinned oracle `5559679229bc961848b121ccdeaa8fa5d79bec98` DOES build, install, import and GENERATE TOKENS inside an `rc` lease on `dgx:gpu0` (2026-08-18), which kills the "a model run is untested" clause those sites carried, and #1213 killed the "a lease cannot produce a runtime" premise underneath it. It survived at `max_num_batched_tokens` 512, `max_model_len` 512 and `gpu_memory_utilization` 0.30 on a ~20 GiB model, where the recorded denominator for this family is 8192 and 2048; `AGENTS.md` §Gates requires vLLM's PRODUCTION configuration as the denominator, so a reduced-`mnbt` arm is a different engine setup rather than a smaller measurement, and `gpu_memory_utilization` is a REFUTED lever (`.agents/specs/mtp-k-gt-1.md`: 0.75 thrashed 42 minutes, 0.30 rebooted the box). The named next levers are `max_num_batched_tokens` and `cudagraph_capture_sizes`, one at a time. The second half is the bytes: `r0b0tlab/Qwen3.8-27B-NVFP4-MTP-sm121`@`36f717a2` is ~20.4 GiB over four shards and is not mirrored where a lease can read it, which is also why its sha256 is recorded as unpaid. Same shape as [#1613](https://github.com/mudler/vllm.cpp/issues/1613) for the block-wise FP8 gate. FIXED IN FLOW: all five citations now name this issue, and the loader is untouched -- W5's accounting and cross-check need no lease and no oracle. Spec [`qwen38-27b-quant-arms.md`](specs/qwen38-27b-quant-arms.md), parent [#821](https://github.com/mudler/vllm.cpp/issues/821) | gap | | [#1538](https://github.com/mudler/vllm.cpp/issues/1538) | `SPEC-DFLASH2` | **vllm#52816's head moved a THIRD time (`66e5414c` -> `3406ec1d`) while it is still open, and refactors `compute_candidates` into `LogitsProcessor.get_top_k_tokens`.** Measured 2026-08-21 by W6 from `raw.githubusercontent.com` at both heads: +11/-80 on `qwen3_dflash2.py`, +4/-16 on `dflash2/speculator.py`, +2/-5 on the base `speculator.py`. The big one is a RELOCATION rather than new math -- the padding mask, the id rebase, the TP all-gather and the scale-THEN-softcap order all survive in `logits_processor.py:241-286`, so `## Owed` O16's reading of the codebook-span question holds at BOTH heads. What does NOT survive is the explicit `UnquantizedEmbeddingMethod`/`UnquantizedLinearMethod` guard that `## Risks/decisions` D12 ports as `RefuseQuantizedDflash2LmHead`, which is deleted at `3406ec1d`; our guard's own reason (the GGUF arm dequantizes `output.weight` to bf16, and a GGUF target with a safetensors DFlash2 draft is admitted here) is independent of upstream's and stands. NOT reconciled in flow, deliberately: `## Gates` G2 fixes the gate head at `66e5414c` while the pull request is unmerged, and moving the port onto a third unmerged head during the gate would move the thing being measured. Owed under `## Owed` O21 of [the DFlash2 spec](specs/dflash2-spec-decode.md) | verification | | [#1456](https://github.com/mudler/vllm.cpp/issues/1456) | `SPEC-DFLASH2` | **The GB10 oracle DOES have a FLASH_ATTN denominator: the arch measurement stands, the conclusion drawn from it does not.** Measured 2026-08-21 by W6 on `dgx:gpu0` through an `rc` lease, with the very wheel #1456 was filed about (`vllm-0.1.dev1+g66e5414c6`, sha256 `fbc247ab...`). A capture that exported `VLLM_ATTENTION_BACKEND=TRITON_ATTN` got `FLASH_ATTN` anyway and RAN: `Using FlashAttention version 2`, 54.87 GiB loaded, CUDA graphs captured (PIECEWISE 5/5, FULL 1/1, plus the DFlash2 speculator's own), 4 x 64 coherent tokens, speculation live at 209 accepted of 350 drafted and mean acceptance length 5.00. No `cudaErrorUnsupportedPtxVersion`. Consistent with the `sm_80`/`sm_75` SASS finding rather than contradicting it: `sm_80` PTX JITs FORWARD, and that error is the OPPOSITE failure (PTX newer than the driver). So `FA_USABLE=0` in the staged `FA-CONSTRAINT.txt` was inferred from emitted arches, never observed from a run, and is the thing to reconcile. A SECOND trap found in the same run and recorded so nobody repeats it: **`VLLM_ATTENTION_BACKEND` does not exist at this revision** -- grepping every `.py` in the wheel returns nothing; the knob is `EngineArgs.attention_backend` (`arg_utils.py:706`) folded into `AttentionConfig.backend` (`:2382`), so the old export selects NOTHING and auto-selection wins silently, letting a run record one backend while executing another. NOT reconciled in flow: W6 does not substitute a denominator the developer declared, and takes both arms instead, each named in its own golden. Owed under `## Owed` O22 of [the DFlash2 spec](specs/dflash2-spec-decode.md) | verification | diff --git a/docs/ENVIRONMENT.md b/docs/ENVIRONMENT.md index d9a056d0c..d66c6b488 100644 --- a/docs/ENVIRONMENT.md +++ b/docs/ENVIRONMENT.md @@ -207,7 +207,7 @@ portable/reference path. In normal operation leave them unset. | `VT_DFLASH_ATTN_WARP` | off (CUDA) | `=1` falls back to the older per-key warp-reduction block-attention kernel instead of the default chunked reduce-scatter form. Kept for the same-binary A/B that recorded the verdict | | `VT_DFLASH_ATTN_KEYLANE` | off (CUDA) | `=1` selects the one-key-per-lane block-attention form. **MEASURED NEGATIVE and not a tuning knob:** 28.90 s/step against the per-key warp kernel's 18.73 on the same binary (sm_110, MiniMax-H3 512x512/33f, seq 3224), 54% slower, because giving each lane a whole K row makes every K load 32-way scattered. Kept only because it is the experiment that located the real constraint | -| `VT_ADOPT_DEVICE_BYTES` | on (acts only where the backend advertises host-addressable device memory: Vulkan always, Metal and ROCm where the part reports unified memory. MEASURED on Vulkan only) | After a weight is uploaded, re-point its host buffer AT the device allocation instead of keeping a second copy. On a unified box the two copies come out of the same RAM. MEASURED on GB10, Qwen3.6-27B bf16 (50.89 GiB): with the mirror the process reaches **VmRSS 100.759 GiB** and drives the machine to MemAvailable 13.85 / MemFree 1.13 GiB of 119.6 GiB before it has even finished allocating; without it the same load completes at **VmHWM 53.413 GiB**. Qwen3-4B: **16.392 -> 9.607 GiB**. Vulkan allocation is byte-identical either way. `0` is the same-binary A/B back to the two-copy behaviour, and it moves ONLY this lever: the `VT_LOAD_DIRECT_UPLOAD` post-upload source-page release still runs at `0` and on a backend without host-addressable device memory, so the two knobs stay independent. It is an adoption, not a release — the bytes survive at the device address and every reader sees them — so tokens are unchanged either way (`test_opt_paged_engine` on Vulkan is 6/6 token-exact, 96/96, both arms). **REACH AND MEASUREMENT ARE NOT THE SAME SET**, and this row used to read as if they were. The lever is gated on `vt::Backend::DeviceMemoryIsHostAddressable()`, read at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`. That predicate defaults to `false` (`include/vt/backend.h`), so a backend has to opt in, and THREE now do. `VulkanBackend` (`src/vt/vulkan/vulkan_backend.cpp`) returns `true` unconditionally, because every allocation it makes is `HOST_VISIBLE|HOST_COHERENT` and persistently mapped. Since [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477), which moved the portable reference tier off the WIDER `UnifiedMemory()`) `MetalBackend` (`src/vt/metal/metal_backend.mm`) answers `MetalContext::unified_memory()`, i.e. `dev.hasUnifiedMemory`, and `RocmBackend` (`src/vt/rocm/rocm_backend.hip`) answers its own `unified_memory_`, i.e. a managed allocator or an integrated part with `PageableMemoryAccess`. **So the lever ACTS on Apple silicon and on an integrated ROCm part, and nobody has measured it on either** — every number above is GB10 through Vulkan. Read those two as reachable-and-unmeasured rather than as covered by this row's numbers ([#1502](https://github.com/mudler/vllm.cpp/issues/1502)). It is still inert on CUDA and CPU, neither of which overrides the default — and that holds by ABSENCE of an override in `CudaBackend` (`include/vt/backend.h` defaults it `false`), NOT by a test. `tests/vllm/platforms/test_platform.cpp` looks like it pins the CUDA answer and does not: its `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` but its `backend()` returns `vt::GetBackend(DeviceType::kCPU)`, so the `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` at `:329` reads the CPU backend. Nothing in the tree pins the real `CudaBackend` ([#1635](https://github.com/mudler/vllm.cpp/issues/1635)). The CPU backend reports `UnifiedMemory() == true` while this narrower predicate stays `false` — the two properties disagreeing is the whole reason the second one exists | +| `VT_ADOPT_DEVICE_BYTES` | on (acts only where the backend advertises host-addressable device memory: Vulkan always, Metal and ROCm where the part reports unified memory. MEASURED on Vulkan only) | After a weight is uploaded, re-point its host buffer AT the device allocation instead of keeping a second copy. On a unified box the two copies come out of the same RAM. MEASURED on GB10, Qwen3.6-27B bf16 (50.89 GiB): with the mirror the process reaches **VmRSS 100.759 GiB** and drives the machine to MemAvailable 13.85 / MemFree 1.13 GiB of 119.6 GiB before it has even finished allocating; without it the same load completes at **VmHWM 53.413 GiB**. Qwen3-4B: **16.392 -> 9.607 GiB**. Vulkan allocation is byte-identical either way. `0` is the same-binary A/B back to the two-copy behaviour, and it moves ONLY this lever: the `VT_LOAD_DIRECT_UPLOAD` post-upload source-page release still runs at `0` and on a backend without host-addressable device memory, so the two knobs stay independent. It is an adoption, not a release — the bytes survive at the device address and every reader sees them — so tokens are unchanged either way (`test_opt_paged_engine` on Vulkan is 6/6 token-exact, 96/96, both arms). **REACH AND MEASUREMENT ARE NOT THE SAME SET**, and this row used to read as if they were. The lever is gated on `vt::Backend::DeviceMemoryIsHostAddressable()`, read at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`. That predicate defaults to `false` (`include/vt/backend.h`), so a backend has to opt in, and THREE now do. `VulkanBackend` (`src/vt/vulkan/vulkan_backend.cpp`) returns `true` unconditionally, because every allocation it makes is `HOST_VISIBLE|HOST_COHERENT` and persistently mapped. Since [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477), which moved the portable reference tier off the WIDER `UnifiedMemory()`) `MetalBackend` (`src/vt/metal/metal_backend.mm`) answers `MetalContext::unified_memory()`, i.e. `dev.hasUnifiedMemory`, and `RocmBackend` (`src/vt/rocm/rocm_backend.hip`) answers its own `unified_memory_`, i.e. a managed allocator or an integrated part with `PageableMemoryAccess`. **So the lever ACTS on Apple silicon and on an integrated ROCm part, and nobody has measured it on either** — every number above is GB10 through Vulkan. Read those two as reachable-and-unmeasured rather than as covered by this row's numbers ([#1502](https://github.com/mudler/vllm.cpp/issues/1502)). It is still inert on CUDA and CPU, neither of which overrides the default — and that holds by ABSENCE of an override in `CudaBackend` (`include/vt/backend.h` defaults it `false`), NOT by a test. `tests/vllm/platforms/test_platform.cpp` looks like it pins the CUDA answer and does not: its `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` but its `backend()` returns `vt::GetBackend(DeviceType::kCPU)`, so the `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` reads the CPU backend. Nothing in the tree pins the real `CudaBackend` ([#1635](https://github.com/mudler/vllm.cpp/issues/1635)). The CPU backend reports `UnifiedMemory() == true` while this narrower predicate stays `false` — the two properties disagreeing is the whole reason the second one exists | | `VT_QWEN35_ALIAS_HOST_WEIGHTS` | on (Qwen3.5/3.6 family; acts only where the platform advertises `host_memory_is_device_addressable()` — a GB10-class integrated part today) | Hand a dense weight's HOST bytes straight to the device kernel instead of allocating a device copy and uploading into it. On a part whose kernels can dereference host storage that copy buys nothing and costs a second full resident copy of every dense weight out of the same RAM. MEASURED on GB10, `Qwen3.8-2.4T-A95B UD-Q1_0` (369.97 GiB) on `--device cuda`: with the second copy the load completes at 61.20 GiB resident and then exhausts the 119.631 GiB box inside its FIRST forward, **zero decode steps over seven attempts** (issue #1299); with the alias the same run reaches **32/32 decode steps at peak RSS 97.75 GiB**. The instrument counts 60.793 GiB of dense weight aliased rather than duplicated (first-forward totals, at call 1361, where re-homing plateaus) against ~9.2 GiB of misaligned GGUF borrows that decline and still stage. `0` is the same-binary A/B back to the staging behaviour, and it is not only a bisect lever: `src/vllm/model_executor/models/laguna.cpp` records a MEASURED GB10 penalty for reading system-allocated memory from the GPU rather than a `cudaMalloc` allocation, worst on a long-K low-parallelism GEMV, so a decode regression has to be separable from the workload. The substitution is otherwise indistinguishable — the aliased pointer is re-homed to 256 bytes, cuBLASLt's own `CUBLASLT_MATMUL_PREF_MIN_ALIGNMENT_A_BYTES` default, which dominates every explicit pointer gate in the CUDA kernels (the strictest asks 32). A DISCRETE device answers the predicate false and is byte-for-byte unchanged. No effect on CPU, Vulkan, Metal or XPU, whose platforms do not advertise the property | | `VT_LOAD_DIRECT_UPLOAD` | on | Load a weight the device consumes VERBATIM by VIEWING the safetensors mmap (`OwnedBytes::Borrow`, keep-alive on the mapping) instead of copying it into an owned host buffer first, so the device upload reads the file mapping and the load moves those bytes ONCE rather than twice. Only whole-range same-size copies qualify — a transpose, a dtype conversion, a dequant, a concatenation or a load-time repack always takes the copy path, and the helper re-checks `numel * sizeof(dtype) == span` and fails closed to the copy on any mismatch. `0` is the same-binary A/B back to copy-then-upload. Bytes are identical either way, so tokens are unchanged. MEASURED on GB10, Qwen3.6-27B bf16 (50.098 GiB), Vulkan, same binary both arms: the weight-load phase goes **19.27 -> 12.48 s warm** (1.54x) and **52.62 -> 32.75 s cold** (1.61x), load-and-one-token **30.39 -> 22.47 s** warm and **62.98 -> 55.60 s** cold, with every ON leg beating every OFF leg. Total bytes MOVED **100.196 -> 81.260 GiB**: the host materialization pass drops **50.098 -> 31.162 GiB** while the 50.098 GiB device upload is unchanged (the model still has to be uploaded once). 37.8% of this checkpoint qualifies; the rest is merged (qkv, gate_up) or transposed (lm_head) at load and correctly still copies | | `VT_LOAD_STATS` | off | `=1` prints one line per load phase (mmap+header, weights) with its wall time, plus the bytes the load MOVED: `host_copy` (source bytes materialized into an owned host buffer), `borrowed` (source bytes viewed in place by the direct-upload path) and `device_upload` (bytes copied host to device). Diagnostic only; it changes no numerics. Issue #150 |