diff --git a/.agents/issue-index.md b/.agents/issue-index.md index e1f5ba9e5..b72340c04 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -565,6 +565,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1613](https://github.com/mudler/vllm.cpp/issues/1613) | `GATE-QWEN38-27B-FP8-BLOCK` | **The `Qwen/Qwen3.8-27B-FP8` block-wise token gate cannot be taken, because the 28.75 GiB checkpoint is not on the share.** `/mnt/nas_share/rc/ckpt/` holds `qwen3.8-27b-hf`, which is the **bf16** artifact -- no `quantization_config` key, `text_config.dtype = bfloat16` -- and `qwen3.8-q1_0`. Neither is this subject. The share has 3.4 TiB free, so the cost is AUTHORITY: `.agents/developer-preferences.md` authorizes large downloads for the `SPEC-DFLASH2` assets only. Nothing else blocks the gate, and that was not known before: a range-request audit of all 66 shard headers at revision `017b9c7a` shows every one of the 407 `F8_E4M3` tensors has `N % 128 == 0` and `K % 128 == 0`, so the sm120 complete-scale-block refusal (#1453) that makes DSV3's `kv_a_proj_with_mqa` unservable blocks NOTHING here; the ragged GDN `in_proj_a`/`in_proj_b` `[48, 5120]` are `BF16` and named in `modules_to_not_convert`; `weight_scale_inv` ships `BF16` (byte-checked via `data_offsets`, not the label) which `LoadFp8BlockRaw` already widens by value; and the per-layer `layers-.safetensors` naming already resolves through `SelectWeightFiles`. Spec `.agents/specs/gate-qwen38-27b-fp8-block.md`, parent #1189 | gap | | [#1614](https://github.com/mudler/vllm.cpp/issues/1614) | `GATE-QWEN38-27B-FP8-BLOCK` | **Three sites said `Qwen/Qwen3.8-27B-FP8` ships "~400" `modules_to_not_convert` entries, and at revision `017b9c7a` it ships 882** (882 unique, 636 outside the vision tower). The number is the evidence for an ARGUMENT -- it is why `IsFp8BlockProjection` reads the config AND the tensors instead of probing dtypes -- so being wrong by more than 2.2x invites the next reader to re-derive it. No reading of the list produces ~400: the visual entries are duplicated under two naming conventions, so distinct modules are about 759, and half of 882 is 441. Sites: the comment above `IsFp8BlockProjection`, the comment above `Fp8BlockQuantConfig::modules_to_not_convert`, and `.agents/specs/model-fp8-block-weight.md`. The routing itself is correct and no defect in it is asserted; two other claims in the same comment were checked against the checkpoint headers and hold (zero `input_scale` tensors, and the `[96, 40]` block-grid hazard is real). Found while auditing the checkpoint for #1613, fixed in the same flow | bug | | [#1502](https://github.com/mudler/vllm.cpp/issues/1502) | `VT-REFTIER-HOST-ADDRESSABLE` | **`docs/ENVIRONMENT.md` described `VT_ADOPT_DEVICE_BYTES` as Vulkan-only and said it has "No effect on CUDA/CPU/Metal", and [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477)) made both halves false.** That change moved `ReferenceTierEligible` off `UnifiedMemory()` onto `Backend::DeviceMemoryIsHostAddressable()` and added truthful overrides so no backend lost the reference tier, so `MetalBackend` now answers `MetalContext::unified_memory()` and `RocmBackend` answers its `unified_memory_`. The weight loader gates the lever on exactly that predicate, at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`, so the lever ACTS on Apple silicon and on an integrated ROCm part. **The correction is not "add two backend names".** Every number in that row is GB10 through Vulkan, and nobody has measured the lever on either new arm, so the row now separates the backends it is MEASURED on from the backends that merely satisfy the predicate — reach and measurement are different claims and the row read as if the measurement covered the reach. CUDA and CPU stay inert and are unchanged: neither overrides the default `false`, which `tests/vllm/platforms/test_platform.cpp` pins for GB10, and the CPU backend answering `UnifiedMemory() == true` while the narrower predicate stays `false` is the whole reason the two properties are separate. The MEASUREMENT on Metal and integrated ROCm stays owed and is listed under `## Owed` in [`vt-reference-tier-host-addressable.md`](specs/vt-reference-tier-host-addressable.md); it needs an Apple-silicon box or an integrated AMD part | documentation | +| [#1635](https://github.com/mudler/vllm.cpp/issues/1635) | `VT-REFTIER-HOST-ADDRESSABLE` | **[#1620](https://github.com/mudler/vllm.cpp/pull/1620) cited `tests/vllm/platforms/test_platform.cpp` as pinning the CUDA backend's `DeviceMemoryIsHostAddressable()` default, and that test reads the CPU backend.** `FakeUnifiedAddressablePlatform`'s `device_type()` override returns `DeviceType::kCUDA` while its `backend()` override returns `vt::GetBackend(DeviceType::kCPU)`, so the assertion `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` reads CPU and the test's own comment says so. Grep those three names; do not look for a line number, because this index is append-only and a number written HERE is permanent whatever file it points at. **Nothing in the tree pins the real `CudaBackend`.** The CONCLUSION survives — `CudaBackend` declares no override, so it inherits `Backend::DeviceMemoryIsHostAddressable`'s `false` default in `include/vt/backend.h` — but it holds by absence, which is a weaker claim than a pin and must not read as one. No gate could catch this: the assertion passes, and it passes for a reason unrelated to what it claims to measure. The wrong citation landed in three places and one is PERMANENT: the `VT_ADOPT_DEVICE_BYTES` row of `docs/ENVIRONMENT.md` (corrected in this flow), the commit message, and the `#1502` row of this index, which is append-only and can never be edited — which is why this correction needs a row of its own rather than an edit. Both are named by ROW and not by line on purpose: the first draft of this row cited the `#1502` row by LINE NUMBER, and `origin/main` appended rows above it while this branch was open, so the line that draft named had stopped identifying the `#1502` row before this row had shipped even once, and a permanent line number into an append-only file would have been wrong on the day it landed — which is [#844](https://github.com/mudler/vllm.cpp/issues/844)'s rule, committed inside the row that exists to describe this class of defect. No COUNT of the rows that moved it is written here either: that number is read off whatever `origin/main` happened to be merged last, so it rots by the very mechanism this row condemns, and a draft of this row carried one that was already stale by the next merge. Found by post-hoc review of four merged records pull requests, not by a gate. The pull request that lands this row carries NO closing keyword and #1635 stays OPEN: this change corrects the citation and records the gap, while the pin itself needs a CUDA device. OWED: either pin the real `CudaBackend` or state in the record that the default holds unpinned | bug | | [#1629](https://github.com/mudler/vllm.cpp/issues/1629) | `KERNEL-ATTN-DENSE-FLASH` | **`test_check_attention_rung_consistency.py` stored a count of the model tree, so every row on the attention-rung allowlist redded it by doing the thing the allowlist exists for.** `ShippedTreeTests::test_the_population_is_not_empty` asserted `>= 9` against a tree holding exactly 9 `vt::Attention(` sites, so a removing row had zero headroom and no green path: leaving the parked stem redded the floor (`8 not greater than or equal to 9`), and deleting it redded the floor and `test_allowlist_holds_only_the_in_flight_stems` as well -- while the allowlist header explicitly recommends the first of those two. That is the `## Records` shape AGENTS.md names, a measurement of one file stored inside another, and it blocked PR #1579 (#1545) and the LTX-2.5 routing row, which removes two of the three parked stems. FIXED HERE, and NOT by lowering the number, which is the known mute-switch: the floor became `>= 1`, because an empty population means the scanner broke and that is the only thing a raw total can honestly detect, and the guard that a rename cannot slip past stays `test_the_six_deliberate_sites_carry_a_marker`, which pins six sites BY NAME. A case that the stem in a red message names a real source file was added beside it, so a typo in the allowlist is still caught without pinning a count. A second drift lock in the same suite, `assertGreater(excused, 0)`, required the shipped allowlist to stay non-empty forever; it is replaced by two synthetic cases that build their own allowlisted population, so the excused counter is pinned without the shipped tree having to keep a stem parked. Found while landing #1578 and #1579 together -- each green in isolation, main red once both land -- and fixed in the same flow | bug | | [#1631](https://github.com/mudler/vllm.cpp/issues/1631) | — | **A comment-only edit is impossible in any of the 43 `scripts/check-*.py` checkers, so a comment that is measurably false in one cannot be corrected.** `scripts/check-pr-size.py:170` classifies every `scripts/check-*.py` and `scripts/check-*.sh` as a `governance_checker`, and `change_errors` then demands a paired `tests/scripts/test_*.py` change that `executable_evidence` proves goes RED against the BASE checker. A comment-only diff leaves BASE and HEAD semantically identical, so no test can distinguish them and no such evidence can exist. Measured on this row: `ERROR: BASE checker stayed green for 'scripts/check-attention-rung-consistency.py'; changed test is not semantic evidence`, rc=1, with the identical invocation against the parent commit exiting 0. Live cost, three comments in `scripts/check-attention-rung-consistency.py` that ship unrepaired in #1578: `:58-61` says widening to `\bAttention\s*\(` is not the repair because it would match every fast rung, when the reason a wider pattern is not the repair is the function-pointer call it still cannot reach; `:93-96` says the `\b` is what excludes `vt::AttentionDenseFlash(`, when the trailing `\(` is, and the `\b` only excludes a leading identifier character as in `xyvt::Attention(`; `:252-255` says `sites - marked` is not the excused count, when on this tree it is (9 sites, 6 marked, 3 excused). The suite beside them was repaired for #1629, so the tree now contradicts itself across two files in the same directory pair. NOT fixed in the flow that filed it: teaching the guard to tell a comment-only or docstring-only diff from a semantic one changes what the gate accepts, which AGENTS.md `## Changing the rules or a checker` routes to its own row, spec and red-before evidence, and the honest report is therefore a filed gap rather than a comment smuggled in beside an unrelated semantic change. A candidate patch is parked on the issue, and two smaller pre-existing defects in `check-pr-size.py` itself (an incomplete entry-point list at `:370-371`, an unread `SELF_CHECKER` constant at `:376`, which the issue body records as `:378` because a line anchor drifts inside the pull request that writes it) are frozen by the same lock. Owed under `## Owed` in [attention-rung-visibility.md](specs/attention-rung-visibility.md) | bug | | [#1632](https://github.com/mudler/vllm.cpp/issues/1632) | `QUANT-QWEN38-27B-NVFP4-ARM` | **W6's NVFP4 token gate named [#1185](https://github.com/mudler/vllm.cpp/issues/1185) as the authority it waits on, and #1185 closed on 2026-08-18 as local-only** -- it tracked one operator's machines rather than a defect here -- so five sites pointed a reader at an issue that reports "closed" without reporting "cleared": `docs/FEATURES.md`, and the spec's `**Related:**` header, wave table, blockers section, `## Owed` list and `## Now`. **The blocker did not close with the issue, and it is not the one the citations described.** The pinned oracle `5559679229bc961848b121ccdeaa8fa5d79bec98` DOES build, install, import and GENERATE TOKENS inside an `rc` lease on `dgx:gpu0` (2026-08-18), which kills the "a model run is untested" clause those sites carried, and #1213 killed the "a lease cannot produce a runtime" premise underneath it. It survived at `max_num_batched_tokens` 512, `max_model_len` 512 and `gpu_memory_utilization` 0.30 on a ~20 GiB model, where the recorded denominator for this family is 8192 and 2048; `AGENTS.md` §Gates requires vLLM's PRODUCTION configuration as the denominator, so a reduced-`mnbt` arm is a different engine setup rather than a smaller measurement, and `gpu_memory_utilization` is a REFUTED lever (`.agents/specs/mtp-k-gt-1.md`: 0.75 thrashed 42 minutes, 0.30 rebooted the box). The named next levers are `max_num_batched_tokens` and `cudagraph_capture_sizes`, one at a time. The second half is the bytes: `r0b0tlab/Qwen3.8-27B-NVFP4-MTP-sm121`@`36f717a2` is ~20.4 GiB over four shards and is not mirrored where a lease can read it, which is also why its sha256 is recorded as unpaid. Same shape as [#1613](https://github.com/mudler/vllm.cpp/issues/1613) for the block-wise FP8 gate. FIXED IN FLOW: all five citations now name this issue, and the loader is untouched -- W5's accounting and cross-check need no lease and no oracle. Spec [`qwen38-27b-quant-arms.md`](specs/qwen38-27b-quant-arms.md), parent [#821](https://github.com/mudler/vllm.cpp/issues/821) | gap | diff --git a/.agents/specs/vt-reference-tier-host-addressable.md b/.agents/specs/vt-reference-tier-host-addressable.md index fba496975..c79749499 100644 --- a/.agents/specs/vt-reference-tier-host-addressable.md +++ b/.agents/specs/vt-reference-tier-host-addressable.md @@ -296,6 +296,21 @@ is the landed commit message. The two GB10 logs that motivated the row are the backends that merely satisfy the predicate. What stays owed is the MEASUREMENT itself on the two new arms, which needs an Apple-silicon box or an integrated AMD part and cannot be taken here. Owned by this row. +- **Nothing in the tree pins the real `CudaBackend`'s + `DeviceMemoryIsHostAddressable()`, and + [#1635](https://github.com/mudler/vllm.cpp/issues/1635) is OPEN and owned + here.** `tests/vllm/platforms/test_platform.cpp` was cited as that pin and is + not one: `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` + while its `backend()` returns `vt::GetBackend(DeviceType::kCPU)`, so the + `CHECK_FALSE` reads the CPU backend and the fixture's own comment says so. The + conclusion survives by ABSENCE of an override — `CudaBackend` declares none, so + it inherits the base `false` in `include/vt/backend.h` — which is a weaker + claim than a pin and must not read as one. `docs/ENVIRONMENT.md` is corrected + here; the `#1502` row in `.agents/issue-index.md` keeps the wrong citation, + because that index is append-only and can never be edited. What stays owed is + the pin itself: either exercise the real `CudaBackend`, which needs a CUDA + device, or state in the record that the default holds unpinned. Owned by this + row. ## Now diff --git a/docs/ENVIRONMENT.md b/docs/ENVIRONMENT.md index 1303a89a7..d66c6b488 100644 --- a/docs/ENVIRONMENT.md +++ b/docs/ENVIRONMENT.md @@ -207,7 +207,7 @@ portable/reference path. In normal operation leave them unset. | `VT_DFLASH_ATTN_WARP` | off (CUDA) | `=1` falls back to the older per-key warp-reduction block-attention kernel instead of the default chunked reduce-scatter form. Kept for the same-binary A/B that recorded the verdict | | `VT_DFLASH_ATTN_KEYLANE` | off (CUDA) | `=1` selects the one-key-per-lane block-attention form. **MEASURED NEGATIVE and not a tuning knob:** 28.90 s/step against the per-key warp kernel's 18.73 on the same binary (sm_110, MiniMax-H3 512x512/33f, seq 3224), 54% slower, because giving each lane a whole K row makes every K load 32-way scattered. Kept only because it is the experiment that located the real constraint | -| `VT_ADOPT_DEVICE_BYTES` | on (acts only where the backend advertises host-addressable device memory: Vulkan always, Metal and ROCm where the part reports unified memory. MEASURED on Vulkan only) | After a weight is uploaded, re-point its host buffer AT the device allocation instead of keeping a second copy. On a unified box the two copies come out of the same RAM. MEASURED on GB10, Qwen3.6-27B bf16 (50.89 GiB): with the mirror the process reaches **VmRSS 100.759 GiB** and drives the machine to MemAvailable 13.85 / MemFree 1.13 GiB of 119.6 GiB before it has even finished allocating; without it the same load completes at **VmHWM 53.413 GiB**. Qwen3-4B: **16.392 -> 9.607 GiB**. Vulkan allocation is byte-identical either way. `0` is the same-binary A/B back to the two-copy behaviour, and it moves ONLY this lever: the `VT_LOAD_DIRECT_UPLOAD` post-upload source-page release still runs at `0` and on a backend without host-addressable device memory, so the two knobs stay independent. It is an adoption, not a release — the bytes survive at the device address and every reader sees them — so tokens are unchanged either way (`test_opt_paged_engine` on Vulkan is 6/6 token-exact, 96/96, both arms). **REACH AND MEASUREMENT ARE NOT THE SAME SET**, and this row used to read as if they were. The lever is gated on `vt::Backend::DeviceMemoryIsHostAddressable()`, read at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`. That predicate defaults to `false` (`include/vt/backend.h`), so a backend has to opt in, and THREE now do. `VulkanBackend` (`src/vt/vulkan/vulkan_backend.cpp`) returns `true` unconditionally, because every allocation it makes is `HOST_VISIBLE|HOST_COHERENT` and persistently mapped. Since [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477), which moved the portable reference tier off the WIDER `UnifiedMemory()`) `MetalBackend` (`src/vt/metal/metal_backend.mm`) answers `MetalContext::unified_memory()`, i.e. `dev.hasUnifiedMemory`, and `RocmBackend` (`src/vt/rocm/rocm_backend.hip`) answers its own `unified_memory_`, i.e. a managed allocator or an integrated part with `PageableMemoryAccess`. **So the lever ACTS on Apple silicon and on an integrated ROCm part, and nobody has measured it on either** — every number above is GB10 through Vulkan. Read those two as reachable-and-unmeasured rather than as covered by this row's numbers ([#1502](https://github.com/mudler/vllm.cpp/issues/1502)). It is still inert on CUDA and CPU, neither of which overrides the default: `tests/vllm/platforms/test_platform.cpp` pins `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())`, and the CPU backend reports `UnifiedMemory() == true` while this narrower predicate stays `false` — the two properties disagreeing is the whole reason the second one exists | +| `VT_ADOPT_DEVICE_BYTES` | on (acts only where the backend advertises host-addressable device memory: Vulkan always, Metal and ROCm where the part reports unified memory. MEASURED on Vulkan only) | After a weight is uploaded, re-point its host buffer AT the device allocation instead of keeping a second copy. On a unified box the two copies come out of the same RAM. MEASURED on GB10, Qwen3.6-27B bf16 (50.89 GiB): with the mirror the process reaches **VmRSS 100.759 GiB** and drives the machine to MemAvailable 13.85 / MemFree 1.13 GiB of 119.6 GiB before it has even finished allocating; without it the same load completes at **VmHWM 53.413 GiB**. Qwen3-4B: **16.392 -> 9.607 GiB**. Vulkan allocation is byte-identical either way. `0` is the same-binary A/B back to the two-copy behaviour, and it moves ONLY this lever: the `VT_LOAD_DIRECT_UPLOAD` post-upload source-page release still runs at `0` and on a backend without host-addressable device memory, so the two knobs stay independent. It is an adoption, not a release — the bytes survive at the device address and every reader sees them — so tokens are unchanged either way (`test_opt_paged_engine` on Vulkan is 6/6 token-exact, 96/96, both arms). **REACH AND MEASUREMENT ARE NOT THE SAME SET**, and this row used to read as if they were. The lever is gated on `vt::Backend::DeviceMemoryIsHostAddressable()`, read at both `AdoptDeviceBytesAsHost` branches in `src/vllm/model_executor/models/qwen3_5_weights.cpp`. That predicate defaults to `false` (`include/vt/backend.h`), so a backend has to opt in, and THREE now do. `VulkanBackend` (`src/vt/vulkan/vulkan_backend.cpp`) returns `true` unconditionally, because every allocation it makes is `HOST_VISIBLE|HOST_COHERENT` and persistently mapped. Since [`cffe59b02`](https://github.com/mudler/vllm.cpp/commit/cffe59b02) ([#1477](https://github.com/mudler/vllm.cpp/issues/1477), which moved the portable reference tier off the WIDER `UnifiedMemory()`) `MetalBackend` (`src/vt/metal/metal_backend.mm`) answers `MetalContext::unified_memory()`, i.e. `dev.hasUnifiedMemory`, and `RocmBackend` (`src/vt/rocm/rocm_backend.hip`) answers its own `unified_memory_`, i.e. a managed allocator or an integrated part with `PageableMemoryAccess`. **So the lever ACTS on Apple silicon and on an integrated ROCm part, and nobody has measured it on either** — every number above is GB10 through Vulkan. Read those two as reachable-and-unmeasured rather than as covered by this row's numbers ([#1502](https://github.com/mudler/vllm.cpp/issues/1502)). It is still inert on CUDA and CPU, neither of which overrides the default — and that holds by ABSENCE of an override in `CudaBackend` (`include/vt/backend.h` defaults it `false`), NOT by a test. `tests/vllm/platforms/test_platform.cpp` looks like it pins the CUDA answer and does not: its `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` but its `backend()` returns `vt::GetBackend(DeviceType::kCPU)`, so the `CHECK_FALSE(gb10.backend().DeviceMemoryIsHostAddressable())` reads the CPU backend. Nothing in the tree pins the real `CudaBackend` ([#1635](https://github.com/mudler/vllm.cpp/issues/1635)). The CPU backend reports `UnifiedMemory() == true` while this narrower predicate stays `false` — the two properties disagreeing is the whole reason the second one exists | | `VT_QWEN35_ALIAS_HOST_WEIGHTS` | on (Qwen3.5/3.6 family; acts only where the platform advertises `host_memory_is_device_addressable()` — a GB10-class integrated part today) | Hand a dense weight's HOST bytes straight to the device kernel instead of allocating a device copy and uploading into it. On a part whose kernels can dereference host storage that copy buys nothing and costs a second full resident copy of every dense weight out of the same RAM. MEASURED on GB10, `Qwen3.8-2.4T-A95B UD-Q1_0` (369.97 GiB) on `--device cuda`: with the second copy the load completes at 61.20 GiB resident and then exhausts the 119.631 GiB box inside its FIRST forward, **zero decode steps over seven attempts** (issue #1299); with the alias the same run reaches **32/32 decode steps at peak RSS 97.75 GiB**. The instrument counts 60.793 GiB of dense weight aliased rather than duplicated (first-forward totals, at call 1361, where re-homing plateaus) against ~9.2 GiB of misaligned GGUF borrows that decline and still stage. `0` is the same-binary A/B back to the staging behaviour, and it is not only a bisect lever: `src/vllm/model_executor/models/laguna.cpp` records a MEASURED GB10 penalty for reading system-allocated memory from the GPU rather than a `cudaMalloc` allocation, worst on a long-K low-parallelism GEMV, so a decode regression has to be separable from the workload. The substitution is otherwise indistinguishable — the aliased pointer is re-homed to 256 bytes, cuBLASLt's own `CUBLASLT_MATMUL_PREF_MIN_ALIGNMENT_A_BYTES` default, which dominates every explicit pointer gate in the CUDA kernels (the strictest asks 32). A DISCRETE device answers the predicate false and is byte-for-byte unchanged. No effect on CPU, Vulkan, Metal or XPU, whose platforms do not advertise the property | | `VT_LOAD_DIRECT_UPLOAD` | on | Load a weight the device consumes VERBATIM by VIEWING the safetensors mmap (`OwnedBytes::Borrow`, keep-alive on the mapping) instead of copying it into an owned host buffer first, so the device upload reads the file mapping and the load moves those bytes ONCE rather than twice. Only whole-range same-size copies qualify — a transpose, a dtype conversion, a dequant, a concatenation or a load-time repack always takes the copy path, and the helper re-checks `numel * sizeof(dtype) == span` and fails closed to the copy on any mismatch. `0` is the same-binary A/B back to copy-then-upload. Bytes are identical either way, so tokens are unchanged. MEASURED on GB10, Qwen3.6-27B bf16 (50.098 GiB), Vulkan, same binary both arms: the weight-load phase goes **19.27 -> 12.48 s warm** (1.54x) and **52.62 -> 32.75 s cold** (1.61x), load-and-one-token **30.39 -> 22.47 s** warm and **62.98 -> 55.60 s** cold, with every ON leg beating every OFF leg. Total bytes MOVED **100.196 -> 81.260 GiB**: the host materialization pass drops **50.098 -> 31.162 GiB** while the 50.098 GiB device upload is unchanged (the model still has to be uploaded once). 37.8% of this checkpoint qualifies; the rest is merged (qkv, gate_up) or transposed (lm_head) at load and correctly still copies | | `VT_LOAD_STATS` | off | `=1` prints one line per load phase (mmap+header, weights) with its wall time, plus the bytes the load MOVED: `host_copy` (source bytes materialized into an owned host buffer), `borrowed` (source bytes viewed in place by the direct-upload path) and `device_upload` (bytes copied host to device). Diagnostic only; it changes no numerics. Issue #150 |