Skip to content

test(VT-REFTIER-HOST-ADDRESSABLE): check CUDA's host-addressability answer in the two halves it actually has - #1778

Merged
localai-bot merged 3 commits into
mainfrom
row/VT-REFTIER-CUDA-PIN
Aug 23, 2026
Merged

test(VT-REFTIER-HOST-ADDRESSABLE): check CUDA's host-addressability answer in the two halves it actually has#1778
localai-bot merged 3 commits into
mainfrom
row/VT-REFTIER-CUDA-PIN

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

CudaBackend::DeviceMemoryIsHostAddressable() gates the portable CPU reference
tier (src/vt/op_provider.cpp), the weight loader's VT_ADOPT_DEVICE_BYTES
adoption (src/vllm/model_executor/models/qwen3_5_weights.cpp) and the
logits-processor bounce (src/vllm/v1/sample/logits_processor/builtin.cpp).
Being wrong there hands a device pointer to a host memcpy, which is the SIGSEGV
that #844 and #1435 measured. Nothing in the tree held that answer.

The test cited as the pin reads a different backend:
FakeUnifiedAddressablePlatform in tests/vllm/platforms/test_platform.cpp
reports device_type() == kCUDA while its backend() returns
vt::GetBackend(DeviceType::kCPU), so its CHECK_FALSE measures the CPU
backend. #1639 corrected that fixture's comment; this change supplies the pin
the record was still owed.

A runtime pin was rejected on a CI fact, not a preference

No job in .github/workflows/ci.yml has a GPU. cuda-fat-build is the only job
with a CUDA toolchain, it runs the nvidia/cuda:13.3.0-devel container on
ubuntu-latest, it configures -DVLLM_CPP_BUILD_TESTS=OFF and it builds the
vllm target alone. The CUDA registrar returns early when cudaGetDeviceCount
finds no device, so vt::GetBackend(kCUDA) throws on every machine this
project's CI owns. A TEST_CASE reading the real backend would report a skip on
every lane forever, and a skip reads as a pass. That is the shape of evidence
#1635 was filed about, so writing one and calling the debt discharged would
repeat the defect rather than close it.

One claim, two halves, each checked where it executes

The record's claim decomposes, and neither half alone is sufficient.

CudaBackend's answer IS the base default. src/vt/cuda/cuda_backend.cu
gains a static_assert beside the class requiring
decltype(&CudaBackend::DeviceMemoryIsHostAddressable) to be
bool (Backend::*)() const. Taking the address of an inherited member through a
derived class yields a pointer-to-member of the class that DECLARES it, so that
type holds exactly while CudaBackend declares no override of its own, and
becomes bool (CudaBackend::*)() const the moment somebody adds one. It fires on
ANY override, including one returning false, because an override invalidates
the reasoning whatever it returns. cuda-fat-build compiles this translation
unit on every push.

The base default is false. tests/vt/test_backend.cpp gains a Backend
subclass that implements the pure virtuals and deliberately declares no
DeviceMemoryIsHostAddressable. Every other fake in the tree overrides that
method and takes the answer as a constructor argument, so each measures its own
override and none reads the default. This one does, and it answers
UnifiedMemory() == true on purpose so the case cannot pass by the two
predicates happening to agree. Every host lane runs it.

Together they are a mechanical proof of CudaBackend answering false, checked
on surfaces that execute rather than skip.

Mutation evidence

Half one, against nvcc 13.3.33 from the CI container, compiling the real file:

Tree nvcc -std=c++20 -Iinclude -Isrc -c src/vt/cuda/cuda_backend.cu
clean, sha256 d9d11f96... rc=0
+ bool DeviceMemoryIsHostAddressable() const override { return true; } rc=2, static assertion failed at cuda_backend.cu(363)
same override returning false rc=2, identical message
restored, sha256 d9d11f96... rc=0

Half two, include/vt/backend.h default flipped to true and the target
rebuilt:

Tree test_backend -tc="Backend::DeviceMemoryIsHostAddressable defaults to false"
clean, sha256 80edf388... assertions: 3 | 3 passed, rc=0
default false -> true assertions: 3 | 1 passed | 2 failed, rc=1
restored, sha256 80edf388... assertions: 3 | 3 passed, rc=0

Both mutated builds compiled, so neither red is a build failure wearing a test
failure, and both files were restored byte-for-byte and re-verified by sha256
and by re-running the check.

Suites: test_backend, test_backend_cross_device,
test_backend_cross_device_vt_attn_decode_d128, test_cuda_backend,
test_op_provider, test_reference_tier -- 6/6 pass; test_backend alone is
9 cases / 51 assertions.

What is still not held, stated rather than left to be inferred

No CI surface observes a live CudaBackend object answering the question,
because no CI surface has a device. tests/vt/test_cuda_backend.cpp gains that
observation using the skip convention every case in that file already follows,
and on a CPU host it reports assertions: 0 and says in its own skip message
that nothing was observed and where the answer actually lives. It is the
empirical belt to the two structural braces. Citing it as the pin would repeat
#1635 exactly, and both the case and the spec say so.

docs/ENVIRONMENT.md claimed nothing pinned the real CudaBackend, which this
change makes false, so it now names both halves and warns off the wrong
citation. The #1502 row of .agents/issue-index.md keeps the original wrong
citation, because that index is append-only and can never be edited; no row is
edited here.

Fixes #1635

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [Claude Code]

mudler added 3 commits August 23, 2026 09:19
…wer where it can be checked, not where it can only be skipped

#1635 offered two ways to discharge its remaining owed item: give the real
`CudaBackend` a pinning test, or state in the record that the default holds
unpinned. This spec takes the first, in the only shape that runs.

A runtime test was rejected on a CI fact rather than a preference. No job in
`.github/workflows/ci.yml` has a GPU, `cuda-fat-build` configures
`-DVLLM_CPP_BUILD_TESTS=OFF` and builds the `vllm` target alone, and the CUDA
registrar leaves `kCUDA` unregistered when `cudaGetDeviceCount` finds no device.
A `TEST_CASE` reading the real backend would therefore skip on every lane
forever, and a skip reads as a pass, which is the shape of evidence #1635 was
filed about.

The record's claim is instead split into the two halves it actually has, each
checked where it executes: a `static_assert` in `src/vt/cuda/cuda_backend.cu`
that `CudaBackend` declares no override, compiled by `cuda-fat-build`; and a
case in `tests/vt/test_backend.cpp` that the inherited default is `false`, run
by every host lane. What stays unobserved -- a live `CudaBackend` answering on
hardware -- is named as such rather than left to be inferred.

The implementation follows in this branch, so the commit order proves the spec
came first.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [Claude Code]
…nswer in the two halves it actually has

`CudaBackend::DeviceMemoryIsHostAddressable()` gates the portable reference
tier, the `VT_ADOPT_DEVICE_BYTES` adoption and the logits-processor bounce, and
nothing in the tree held it. The test cited as the pin reads a different
backend: `FakeUnifiedAddressablePlatform` reports `device_type() == kCUDA` while
its `backend()` returns `vt::GetBackend(DeviceType::kCPU)`.

A runtime test cannot replace it, and the reason is a CI fact. No job has a GPU;
`cuda-fat-build` is the only job with a CUDA toolchain and it builds with
`-DVLLM_CPP_BUILD_TESTS=OFF`; and the CUDA registrar leaves `kCUDA` unregistered
when `cudaGetDeviceCount` finds no device. A case reading the real backend would
skip on every lane forever, and a skip reads as a pass -- the same shape of
evidence #1635 was filed about.

The claim has two independent halves, and each is now checked where it executes.
`src/vt/cuda/cuda_backend.cu` gains a `static_assert` that
`decltype(&CudaBackend::DeviceMemoryIsHostAddressable)` is
`bool (Backend::*)() const`: taking the address of an inherited member through a
derived class yields a pointer-to-member of the DECLARING class, so that type
holds exactly while `CudaBackend` declares no override. `tests/vt/test_backend.cpp`
gains a `Backend` subclass that declares no override and requires the inherited
default to be `false` -- every other fake in the tree overrides the method and so
measures its own override, never the default.

`tests/vt/test_cuda_backend.cpp` gains the observation on a real device. It
asserts nothing without a GPU and says so in its skip message, so it is the belt
and not the braces.

Mutation-proven on both halves. Adding an override returning `true` to
`CudaBackend` fails `nvcc` 13.3.33 with the intended message (rc=2), an override
returning `false` fails it identically, and the restored file recompiles clean at
the same sha256. Flipping the base default in `include/vt/backend.h` to `true`
turns the new case red (2 of 3 assertions, rc=1), and the restored header is
byte-identical.

`docs/ENVIRONMENT.md` said nothing pinned the real `CudaBackend`, which this
change makes false, so it now names both halves and warns off the wrong citation.

Refs #1635

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [Claude Code]
Keeps the branch current before the gate run; no conflict.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [Claude Code]
@localai-bot
localai-bot merged commit 775c1f6 into main Aug 23, 2026
22 of 25 checks passed
@localai-bot
localai-bot deleted the row/VT-REFTIER-CUDA-PIN branch August 23, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The unified-memory record cites a test that reads the CPU backend, not CUDA

2 participants