docs(FIX-OFFLOAD-DOCS-957): weight offload is refused at startup, not accepted and inert (#957) - #958
Merged
Merged
Conversation
… accepted and inert Closes #957. No code moves. `4a183b731` (#887) turned a configured weight offload from ACCEPTED-AND-INERT into a hard startup refusal, and neither public document followed. Both still described the behaviour that commit replaced, so a reader who followed them would add `--offload-config` expecting a no-op and get a server that does not start. ## What the engine actually does `RefuseUnsupportedWeightOffload` (`weight_offloader.cpp:72-83`) throws when offloading is enabled and the resolved architecture does not declare support. It is called unconditionally from the load path (`model_loader.cpp:1410-1414`), right after `ModelRegistry::Resolve` and BEFORE any weight I/O. `ModelFactory::supports_weight_offload` defaults to false and NO model in the tree sets it, so EVERY architecture is refused today. The tree asserts this itself: `test_weight_offloader.cpp:376-379` counts the declaring models and carries the message "a model now declares supports_weight_offload; update this". Negative control on the grep: a misspelled needle returns 0, so the empty set is not a wrong search term. A second refusal from that commit was also undocumented — `VerifyWeightOffloadWasConsulted` throws after load when a model declares support and then consults the offloader about zero weights, reported as a defect in that loader rather than a configuration error. The old refusal list covered config validation only. ## Checked, not assumed The example error names `Qwen3MoeForCausalLM`, which is a real registered architecture (`qwen3_moe_registry.cpp:181`). An invented name in an error example is indistinguishable from a real one to a reader. The "only when enabled" claim is `weight_offloader.cpp:75`. The quoted message is transcribed from the throw. `docs/STATUS.md` needed no change: it already lists weight offload under what is owed and never claimed the inert behaviour. ## THIS DOES NOT MAKE documentation-checkpoint GREEN `check-doc-checkpoint.py` evaluates each commit against its OWN parent, so no later commit can satisfy an earlier one, and there is no waiver mechanism (`:256` says exempting named paths was deliberately avoided). Measured: `docs/FEATURES.md`/`docs/USAGE.md` changed in 66 commits since `73d217db`, which still reports ERROR. The 28-commit backlog is closed only by advancing the base or changing the checker, and this row takes neither decision. ## The other 27 owed nothing Audited all 28. The 19 `MODEL-MM-indextts2` commits are intermediate waves whose campaign documented itself at `f374ab8ed` once the pipeline rendered end to end, and every one is an ancestor of it. The 3 `ENG-EXPERT-STREAM` commits are deliberately unwired staged slices that `docs/FEATURES.md` must NOT claim — #918's own body says "Nothing wired to a loader". Four earlier `ENG-WEIGHT-OFFLOAD` commits predate `62406c30e`, which documented them. `fba312c67` is a record/measurement commit shipping no user surface. A staged campaign that documents itself once, when the capability becomes reachable, is the CORRECT shape. The gate flags every wave that is not that commit, so a large flagged count is expected and is not by itself evidence of stale docs. Only a commit landing user-visible behaviour AFTER its campaign's docs can make a document wrong. Exactly one of the 28 did. Spec: `.agents/specs/offload-docs-refusal.md`. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
localai-bot
pushed a commit
that referenced
this pull request
Aug 15, 2026
FOLLOWING_AGENTS_PROTOCOL Keeps the branch current before review. No content of this row changes; the incoming commit is the weight-offload docs fix (#958). Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This was referenced Aug 15, 2026
localai-bot
added a commit
that referenced
this pull request
Aug 16, 2026
…rim it back (#930) (#964) The LTX-2.5 phase loop was fixed at one `Ltx2VideoTokenCount(vshape, 1)`, and that single limitation blocked three conditioning arms. Two rows walked into it hours apart without talking to each other — [#930](#930) from the IC-LoRA port and [#920](#920) from the generated-keyframe-slots port — which is why it gets a seam of its own rather than a fix inside whichever row noticed it last. Issue: [#930](#930). Spec: [`.agents/specs/ltx25-token-append.md`](.agents/specs/ltx25-token-append.md). Campaign [#644](#644). Pin `Lightricks/LTX-2 @ fd4ded7f2d88d3da713abcdd4ad41ecc4a9314ca`, verified at the working checkout. ## Why this branch replaces #948 **This is a history repair, not a content one.** [#948](#948) carried this row through implementation and a fresh review to PASS at head `a1b1d8da4`. It cannot land, and the reason is entirely in its commit ORDER. `scripts/check-doc-checkpoint.py` walks a range **per commit** ([#573](#573)). The review-repair commit `b5618b305` edited `include/vllm/model_executor/models/ltx2_conditioning.h` and `include/vllm/multimodal/ltx2_video.h` — the `user_usage` class — while the `docs/USAGE.md` edit that pays for them sat in a sibling commit. The final tree satisfied the obligation; only the ordering failed. Measured on this branch, the checker still REDs on `b5618b305` in isolation and is green on both commits here, so the instrument is armed rather than merely quiet. A commit message cannot be corrected in place and this repository uses no force variant, so the branch is rebuilt with the shape the checker requires: **one spec commit, then one implementation commit** carrying `docs/USAGE.md` and `docs/FEATURES.md` alongside the code they describe. #678 was rebuilt as #880 and #661 as #882 for exactly this rule. **The reviewed content is proven unchanged.** Built on `2daa3287f` — `origin/main` when the rebuild started — this branch's head tree hash was `d937252dd06efd6a07c4affebff42153b25075a5`, byte-for-byte equal to `a1b1d8da4^{tree}`, and `git diff a1b1d8d HEAD` was empty. `origin/main` then advanced to `f365cc299` (#958), so the branch was rebased onto it; restricted to this row's ten files, the only remaining differences against `a1b1d8da4` are the two lines #958 itself changed — its own appended issue-index row and its `--offload-config` rewrite in `docs/USAGE.md`. The eight code, test, spec and `docs/FEATURES.md` files are byte-identical. Nothing below was re-derived. ## The sizing in #930 is corrected, not inherited #930 names two blocked arms. There are three: reference video, the last-frame keyframe, and generated keyframe slots. #930 predates #920 and its author asked for the correction. This spec is the record that supersedes both counts. ## What was actually missing The appending conditioning ITEMS were already ported and gated. Two halves were not. `Ltx2ExtendKeyframesMask` mirrors `extend_keyframes_mask` (`conditioning/mask_utils.py:76-105`). Upstream's docstring makes the call an obligation of appending itself — *"Every conditioning item that appends tokens must call this, otherwise the per-token marker goes out of sync with the token sequence"* (`:83-85`) — so it lives inside `AppendTokens` rather than at the three call sites. Both `None` branches are mirrored, including the zero-fill only `marked=true` reaches (`:100-101`), so #920's arm finds it already correct. `Ltx2ClearConditioning` mirrors `clear_conditioning` (`tools.py:88-117`). Two things there are not a truncation: the denoise mask comes back **all ones** rather than sliced (`:104`), because the returned state describes a finished latent, and `keyframes_mask` is dropped entirely (`:113`). `Ltx2LatentState` grows a `keyframes_mask` field so the marker can be extended with the sequence it describes. ## Three findings that changed the design **The attention mask is not the gap, and no field is added for it.** Both appending video items pass a literal `attention_mask=None` (`keyframe_cond.py:70`, `reference_video_cond.py:88`), `update_attention_mask` returns `None` for that case (`mask_utils.py:110-143`), and the only route to a non-`None` mask is `ConditioningItemAttentionStrengthWrapper`, whose sole application site is `iclora_utils.py:169` on the IC-LoRA path. `combined_image_conditionings` — the route this engine mirrors — never wraps. A field here would be one no ported item could populate, which is the unpassed-parameter shape [`.agents/reachability.md`](.agents/reachability.md) enumerates. The refusals that cite the absent field were right to mention it and wrong to offer it as the blocker. **The sigma schedule must keep reading the target count.** Upstream fixes it twice over: the argument is `math.prod(latent.shape[2:])` of the *unpatchified* target (`schedulers.py:32`), which by construction cannot contain appended tokens, and every pipeline computes its sigmas before a state exists (`ti2vid_one_stage.py:207` passes no latent; `distilled.py:200-201` uses frozen constants). This engine's call sits **after** the conditioning block, so before this change it would have re-shifted the whole trajectory the moment anything appended. It now reads a `target_tokens` local. **The trim cannot be gated on pixels alone, so it is gated on a guard.** Appended tokens sit at the tail of a contiguous `[tokens, width]` buffer and `Ltx2VideoUnpatchify` takes a bare pointer, so an un-trimmed state would unpatchify the same head bytes and render pixel-identical frames. Correct output for the wrong reason. A `VT_CHECK` at the pointer boundary is what turns "the head happens to be right" into "the buffer *is* the target grid", and it is what makes deleting the trim a RED (mutation M3). ## The arm lifted, and the two left alone The **last-frame keyframe** arm is served, mirroring `combined_image_conditionings` (`helpers.py:272-308`): one preprocess-and-encode shared by both arms, and a branch on `frame_idx` sending 0 to `VideoConditionByLatentIndex` (`:296`) and anything else to `VideoConditionByKeyframeIndex` (`:302`). Both may be supplied at once. Nothing is added to `include/vllm.h` — `last_frame` was already on the ABI. **Reference video stays refused and its message is byte-identical.** At this base, PR #938 (`row/LTX25-IC-LORA`) is open and unmerged and `git log --grep '#923'` is empty, so `--lora` does not read the IC-LoRA scale factors and that refusal's stated cause is still true. #930's body describes the message as already rewritten onto token-append; **that is not the state of `origin/main`.** Rewording it would have shipped a refusal naming the first of two causes — the defect a sibling row nearly shipped. **Generated keyframe slots** (#920, PR #929, also open) and **reference audio** stay refused for their own reasons. ## What lands unreached, named rather than discovered `Ltx2ExtendKeyframesMask`'s **`marked=true` branch lands with a unit driver and no production caller.** The only upstream construct that passes `true` is `VideoGeneratedKeyframeSlots` (`keyframe_slots.py:121`), which belongs to the generated-keyframe-slots arm and not to this row. Every appending item ported here passes `false`. It is landed rather than stubbed because building half of `extend_keyframes_mask` is the worse option: the two `None` branches are not symmetric, and a port that mirrors one and guesses the other gets #920's arm silently wrong. Mutation M5 shows the branch is not inert — appending with `marked=true` instead of `false` is RED. Owner of the wiring: row `LTX25-GENERATED-KEYFRAMES`, issue [#920](#920). Listed under `## Owed` in the spec. Declared here per [`.agents/reachability.md`](.agents/reachability.md) `## Landing a slice that is not reached yet`. ## Evidence RED first, on the pre-row engine with the tests in place: `test_ltx2_video` FAILED, the new case throwing the last-frame refusal verbatim. That run also printed `assertions: 825 | 825 passed | 0 failed` beside `Status: FAILURE!` — the thrown-case trap, which is why the exit code is the authority throughout. The witness is on **rendered artifact bytes with a no-op control**, because `Ltx2ConditioningTrace` is filled before denoise and cannot observe the loop. A keyframe render differs from a no-keyframe render in **59773 of 91169** bytes; two different keyframes differ from each other in **334**; the same request twice is byte-identical. The control is what separates "the append reached the maths" from "the instrument is blind" — a sibling row's first attempt found every arm identical *including* its control, and without that control a blind instrument reads as a real-but-subtle effect, which is the direction that ships. Two new trace fields, `video_tokens` and `schedule_tokens`, are written *inside* the phase loop for that same reason, and the sigma claim is checked as a relation between two measured values rather than against a literal. Upstream ships **no tests** — `find /home/mudler/_git/LTX-2 -name 'test_*.py'` returns 0 across the whole repository at the pin — so nothing was ported and every case is written against an upstream anchor instead. ## Review repairs A fresh review returned FAIL on six findings. The seam itself survived: every upstream design decision checked out and the pixel witness reproduced exactly. Two claimed guarantees survived mutation and are now gated. **The all-ones restore was not gated.** `Ltx2CreateVideoLatentState` already sets every target-range mask value to `1.0`, and the append writes `1 - strength` only at the tail, which the trim drops under either implementation — so "restore all ones" and "slice" produced identical bytes over the range the test walked, and mutation M6 (slice instead of restore) was **GREEN**. The case now conditions token 0 *inside* the target with `Ltx2ConditionVideoByLatentIndex` before the append, which writes `1 - strength` at `start .. start + count` (`latent_cond.py:41`). A slice then leaves `0.4` at token 0. M6 is **RED**, and a `REQUIRE` on `mask.front()` before the trim asserts the instrument is armed rather than assuming it. **Nothing gated that the keyframe lands on the last frame.** Mutation M10 (`frame_idx` from `frames - 1` to `0`) was **GREEN**: both renders still differed from the no-op control and from each other, and the token count was identical, because a keyframe pinned to the first frame appends exactly as many tokens as one pinned to the last. The engine now asserts the first appended token's temporal position, recomputed from `frames` and `fps` rather than read back from the argument, so the two are independent expressions (`keyframe_cond.py:52-58`). M10 is **RED**. `origin/main` advanced to `c2019b0e3` (#935) during review and the branch stopped merging. Three conflicts, two with a trap: `docs/FEATURES.md`'s LTX-2.5 cell is at `MAX_CELL_CHARS` exactly, so the obvious merge lands at 226 and REDs — resolved by keeping main's complete cell and trimming only this row's own wording, to 220. And both sides had rewritten the `READER ANCHORS` line, so **neither** value was correct afterwards; it was re-derived a third time from the merged file. Every upstream `file:line` in this change was re-derived at the final tree by a needle taken from the **claim**, never read out of the cited span, and the load-bearing needles were checked for uniqueness in their files. The sweep covers **119** citations in this row's added lines — 108 explicit and 11 bare `:NN` continuations — and corrected **13**. Four of those were the review's: `tools.py:103` and `:112` in the `Ltx2ClearConditioning` declaration and in the spec, and `schedulers.py:38-39` in four places including the append-only issue-index row, which is fixed first and SHA-anchored because it could not be corrected after landing. Three were not, and two of those three are anchors written *during this repair* and caught by the sweep over its own output: - `ltx2_recipes.py:125-158` names a file that exists neither upstream nor in this tree. The distilled recipe's frozen sigmas are `distilled.py:200-201` over `utils/constants.py:17-23`. - A bare `(:100-101)` resolves against `tools.py`, the nearest file named above it, whose `:100-101` is a real and unrelated statement. It means `mask_utils.py`. The two bare continuations in `Ltx2ClearConditioning`'s declaration are spelled with their file for the same reason: the nearest name above them is `blocks.py`, and `blocks.py:104` and `:113` are both real imports. - `latent_cond.py:36-38`, written here for the M6 repair, is `stop_token` and a `clone()`. The `1 - strength` write is `:41`. **One anchor is left alone deliberately.** `latent_cond.py:38-39` is cited at eight pre-existing sites across specs, golden scripts and other rows' tests for "only `clean_latent` and `denoise_mask` are written". Those writes are `:40-41`; `:38-39` is a `clone()` and a blank line. The drift predates this row and the sites belong to other rows, so correcting them here would be scope this PR did not claim. It is named rather than left for the next sweep to rediscover. ## Gate on the rebuilt branch Rerun in full rather than inherited, on the rebased head `fd9948f14`, CPU-only Release, `-j6` build and `ctest -j4`: `CONFIGURE_EXIT=0`, `BUILD_EXIT=0`, `: error:` count **0**, `No space left` **0**, `BFD assertion` **0** (all three greps positive-controlled against a synthetic line), `ctest -N` **485**, `CTEST_EXIT=0`, `100% tests passed, 0 tests failed out of 485`. Load average 4.94 at start and 11.02 at end; 46 G free. Row suites, run directly for their assertion counts: `test_ltx2_video` 44/44 cases, 1024/1024 assertions; `test_ltx2_vae` 39/39, 3092/3092; `test_ltx2` 43/43, 4388/4388; `test_ltx2_device` 18/18, 546/546. M6 and M10 were rerun on this branch rather than carried over. **M6** — `Ltx2ClearConditioning` slices instead of restoring — `git diff --stat` one file `1 +/1 -`, BUILT with 0 compile errors, exit **1**: `38 passed | 1 failed`, four `CHECK( 0.4 == 1 )` at mask tokens 0-3. **M10** — `frame_idx` `frames - 1` to `0` — one file `1 +/1 -`, BUILT with 0 compile errors, exit **1**: `42 passed | 2 failed`, both cases THREW the temporal-position `VT_CHECK` (`0.333333` wanted, `0.000000` got). M10 prints `assertions: 923 | 923 passed | 0 failed` beside `Status: FAILURE!`, which is the thrown-case trap; the exit code is the authority. The tree was restored byte-for-byte after each — `git write-tree` back to `d937252dd0` — and rebuilt before remeasuring. `READER ANCHORS` were re-derived at the final tree and are unchanged: `756 811 907 923 925 1003 1028 1133 1174`. The `docs/FEATURES.md` LTX-2.5 support cell measures **220** characters, exactly `MAX_CELL_CHARS`, with main's `, A2V WAV` and every host and measurement qualifier intact. Checkers, exit codes captured directly: `check-doc-checkpoint` **0** over the range and **0** on each commit individually, `check-commit-trailers` **0**, `check-commit-style` **0**, `check-issue-index-append-only` **0**, `check-agent-record` **0**, `check-public-doc-tables` **0**. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
docs(FIX-OFFLOAD-DOCS-957): weight offload is refused at startup, not accepted and inert
Closes #957. No code moves.
4a183b731(#887) turned a configured weight offload from ACCEPTED-AND-INERTinto a hard startup refusal, and neither public document followed. Both still
described the behaviour that commit replaced, so a reader who followed them
would add
--offload-configexpecting a no-op and get a server that does notstart.
What the engine actually does
RefuseUnsupportedWeightOffload(weight_offloader.cpp:72-83) throws whenoffloading is enabled and the resolved architecture does not declare support. It
is called unconditionally from the load path (
model_loader.cpp:1410-1414),right after
ModelRegistry::Resolveand BEFORE any weight I/O.ModelFactory::supports_weight_offloaddefaults to false and NO model in thetree sets it, so EVERY architecture is refused today. The tree asserts this
itself:
test_weight_offloader.cpp:376-379counts the declaring models andcarries the message "a model now declares supports_weight_offload; update this".
Negative control on the grep: a misspelled needle returns 0, so the empty set is
not a wrong search term.
A second refusal from that commit was also undocumented —
VerifyWeightOffloadWasConsultedthrows after load when a model declaressupport and then consults the offloader about zero weights, reported as a defect
in that loader rather than a configuration error. The old refusal list covered
config validation only.
Checked, not assumed
The example error names
Qwen3MoeForCausalLM, which is a real registeredarchitecture (
qwen3_moe_registry.cpp:181). An invented name in an errorexample is indistinguishable from a real one to a reader. The "only when
enabled" claim is
weight_offloader.cpp:75. The quoted message is transcribedfrom the throw.
docs/STATUS.mdneeded no change: it already lists weight offload under what isowed and never claimed the inert behaviour.
THIS DOES NOT MAKE documentation-checkpoint GREEN
check-doc-checkpoint.pyevaluates each commit against its OWN parent, so nolater commit can satisfy an earlier one, and there is no waiver mechanism
(
:256says exempting named paths was deliberately avoided). Measured:docs/FEATURES.md/docs/USAGE.mdchanged in 66 commits since73d217db, whichstill reports ERROR. The 28-commit backlog is closed only by advancing the base
or changing the checker, and this row takes neither decision.
The other 27 owed nothing
Audited all 28. The 19
MODEL-MM-indextts2commits are intermediate waves whosecampaign documented itself at
f374ab8edonce the pipeline rendered end to end,and every one is an ancestor of it. The 3
ENG-EXPERT-STREAMcommits aredeliberately unwired staged slices that
docs/FEATURES.mdmust NOT claim —#918's own body says "Nothing wired to a loader". Four earlier
ENG-WEIGHT-OFFLOADcommits predate62406c30e, which documented them.fba312c67is a record/measurement commit shipping no user surface.A staged campaign that documents itself once, when the capability becomes
reachable, is the CORRECT shape. The gate flags every wave that is not that
commit, so a large flagged count is expected and is not by itself evidence of
stale docs. Only a commit landing user-visible behaviour AFTER its campaign's
docs can make a document wrong. Exactly one of the 28 did.
Spec:
.agents/specs/offload-docs-refusal.md.FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]