fix(#919): refuse a resolution the latent grid cannot represent, and publish the LTX-2.5 envelope by localai-bot · Pull Request #939 · mudler/vllm.cpp

localai-bot · 2026-08-15T17:38:51Z

Nothing in this tree established what resolutions LTX-2.5 supports. ltx2-gen exposes --width, --height and --frames, and exposing a flag is not supporting a value. Reading the pinned oracle answers the three axes differently, and that difference is the finding.

Closes #919. Row LTX25-RESOLUTION-ENVELOPE, spec .agents/specs/ltx25-resolution-envelope.md. Upstream pin Lightricks/LTX-2 @ fd4ded7f2d88d3da713abcdd4ad41ecc4a9314ca, verified at the local checkout before any anchor was read.

The defect

vllm_video_generate integer-divided width and height into the latent grid and checked only the lower bound. Measured on the reduced fixture at the base SHA, before the guard existed:

Request	Recipe	What happened
width 80	distilled two-stage	rendered 64x64, exit success
width 100	one-stage	rendered 96x64, exit success
width 96	distilled two-stage	threw `the upsampled latent is 4x2x2x2 but phase 'refine' needs 4x2x2x3`

The third is not a silent floor, but it is not a usable error either: stage 1 floors 48 to one latent cell while stage 2 needs three, so the upsampler's shape check fires with a true statement about latents and no mention of the width the caller passed. One guard at the entry point closes both faces, which is why the test carries all three sizes. Our own docs/USAGE.md already documented the divide-by-64 rule as though something enforced it.

Upstream raises, so we refuse

assert_resolution (ltx-pipelines utils/helpers.py:540-551) raises ValueError, from the top of a pipeline's __call__ and before any work is paid for. Nine invocations, including ti2vid_two_stages.py:184 and ti2vid_two_stages_hq.py:199 (is_two_stage=True) against ti2vid_one_stage.py:156 (False).

Nine, and not the ten an earlier revision of this description claimed, nor the twenty-one lines a grep for the name returns: those are 9 invocations + 1 definition + 10 imports + 1 __all__ string. Nor is it every pipeline. 13 pipeline __call__s take a height and a width, and the four that do not call the guard are distilled_mgpu.py:143, ti2vid_two_stages_mgpu.py:163, ti2vid_two_stages_hq_mgpu.py:164 and hdr_ic_lora.py:352. The count was established by walking every def __call__ and reading its signature, because the grep that produced "ten" counts imports.

The divisor is derived, not restated. Upstream spells 64 and 32 as literals chosen by a bool, but that pair is the VAE spatial factor (32, ltx_core/types.py:31-33) times the worst spatial downscale any phase applies: a two-stage pipeline runs stage 1 at width // 2 (ti2vid_two_stages.py:226-228), so a request must survive being halved and still divide the grid. Ltx2AssertResolution takes the divisor as a parameter and the call site computes factors.height * recipe.max_spatial_downscale(), which reproduces upstream's two numbers on the two shipped arms and stays correct for a recipe whose phases downscale further. Two subcases drive the same width 96 through both recipes and require opposite answers, so a hardcoded 64 fails the suite.

One divisor covers both axes, as upstream has it. That is the mirror rather than a simplification, and the call site asserts factors.height == factors.width so a VAE that ever broke the assumption fails by name instead of having its width measured against the height factor.

Frames are the opposite answer

--frames is deliberately left rounding, and the asymmetry is upstream's. resolve_num_frames (utils/blocks.py:908-928) returns an explicit count verbatim (:920-921), and VideoLatentShape.from_pixel_shape (ltx_core/types.py:113) then floors it exactly as this engine does. Adding a refusal there would diverge from the reference rather than mirror it, so the repair is to the document that promised enforcement.

The premise this rested on was false, and the replacement is checkable. An earlier revision said snap_frames_to_grid (utils/helpers.py:554-562) "is reached only from the auto-duration path". It has three callers, not one: utils/helpers.py:581 inside seconds_to_clamped_num_frames, which is the auto-duration path, and dubit.py:215 and :396, the second three lines after DubitPipeline.__call__'s own assert_resolution. What actually holds is sharper and can be checked against a signature rather than a call graph: DubitPipeline.__call__ takes no num_frames parameter at all (dubit.py:194-210) and snaps a count it read from a reference video's container metadata. It is the only pipeline __call__ that snaps, and the only one with no frame-count parameter. Every __call__ that does take one leaves it unsnapped.

The published envelope

docs/USAGE.md gains "The supported resolution envelope", which separates what is legal from what fits: legal sizes are any multiple of 64 or 32; upstream's own defaults are 1024x1536 and, for the HQ preset, 1088x1920 at 121 frames; and the measured ceiling on one GB10 is 320x192 at 25 frames, with 448x256 losing about 59 GB inside the decode. That gap is a decode problem and not a cap. There is no maximum-size check anywhere in this path, and the 60 GB is explicitly not attributed — the decode's own heap peak at that size is 361.72 MiB.

Tests

No upstream test is ported, because there is none to port. Lightricks/LTX-2 at fd4ded7f contains zero test_*.py files anywhere in the repository. The tests are written against upstream anchors instead. Only the provenance changes: each fails for the intended reason before the change, enters through vllm_video_generate rather than constructing the type, and names the upstream file:line justifying the behaviour it asserts.

Reachability. All three production entry points reach the guard, traced by hand and re-derived at the head below rather than carried forward: include/vllm.h:969 vllm_video_generate → src/capi/vllm_c.cpp:1646 engine->engine->Generate(gen); examples/ltx2_gen/main.cpp:320 vllm_video_generate(...) through the same ABI (it includes vllm.h and nothing internal); and the OpenAI /v1/videos route through src/vllm/entrypoints/openai/server_main.cpp:1292 video_engine->Generate(...). All land in Ltx2VideoEngine::Generate (ltx2_video.cpp:1210) and hit the guard at :1547.

Three mutations were re-run at 59d4f59ca, the head this description belongs to, because two of them had been claimed by construction and never executed, and the third had not been re-run since the last merge. Each carries three facts — the diff after applying, whether it BUILT with the compile-error count, and the exit code — because a mutation that fails to build establishes nothing:

Mutation	Diff	Built	Exit	Result
Swap the two axis names in the refusal	1 file, +2/-2	YES, 0 errors	1	RED — 10 of 33 assertions fail
Suggestion back to the bare floor (the `0x64` a sub-divisor caller used to get)	1 file, +2/-6	YES, 0 errors	1	RED — 5 fail, headed by `CHECK(msg.find("0x64") == npos)`
Delete the production call site	1 file, +1/-1	YES, 0 errors	1	RED — the case aborts on its first `FAIL`

The swap is the mutation that mattered, and it was green before this repair. The message emits the literal " (width x height) " label in every refusal it ever produces, so msg.find("width") was satisfied by that constant no matter which axis the guard named: a height-80 request could have reported "the width is not" with nothing to see it. The needles are now whole phrases (the width is not / the height is not / the width and height are not), every subcase asserts the other axis absent, and an 80x80 subcase covers a branch no test executed.

Each mutation was restored by writing the original bytes back and comparing sha256 (identical in all three cases, git status clean), and the tree was rebuilt before any measurement was taken afterwards.

The case name contains a comma, and -tc treats a comma as a filter separator. -tc="*is REFUSED, per recipe*" does not select this case: it selects three unrelated ones, runs 8 assertions instead of 33, and prints SUCCESS. Every run above therefore used -tc="*does not divide the latent grid*" and asserted a non-zero case count; the baseline is 1 case / 33 assertions / exit 0.

Gate

Clean-room: build/ did not exist and was configured from scratch for this run.

Run at this branch's head, 59d4f59ca, on a loaded box — the load is recorded beside every number rather than left to be assumed:

CONFIGURE_EXIT=0                                    loadavg 15.89
BUILD_EXIT=0   ': error:' = 0   'No space left' = 0   'BFD assertion' = 0
ctest -N       Total Tests: 485
CTEST_EXIT=0   100% tests passed, 0 tests failed out of 485   (2 skipped)
Total Test time (real) = 270.86 sec           loadavg 146.38 at start, 50.36 at end

The two skipped are test_modelopt_mixed_precision_checkpoint and test_voxtral_e2e, both skipped by their own guards.

Nothing on this run needed a serial re-run: the load-dependent cases (test_openai_conformance, test_cpu_threadpool, test_engine_core_proc, test_async_llm, test_cpu_x86_llamacpp_floor) all passed under -j4 at loadavg 146.

The three zero counts are measurements, not a broken instrument. Each grep pattern was positive-controlled against a synthetic log carrying all three strings: No space left → 1, BFD assertion → 1, : error: → 0 before the error line was appended and 1 after. The binary was also checked for probe residue (strings … | grep -c PROBE = 1, and that one hit is VT_H3_VAE_PROBE, a committed env-var name in minimax_h3_pipeline.cpp:581, not residue), and its mtime post-dates the rebuild that followed the last mutation restore.

scripts/agent-integration.py --base origin/main reports all gates green locally, including check-agent-record, check-doc-checkpoint, check-now-current, check-public-doc-tables, the trailer suites and the commit-style suites.

check-doc-checkpoint.py walks per commit (#573), so it was run that way on all ten commits in origin/main..HEAD before the push: all ten exit 0. Positive control: --commit b5618b305 exits 1, so the checker was armed.

That per-commit walk found one real defect and it is repaired here. The recovered repair commit edits src/vllm/model_executor/models/ltx2_pipeline.cpp — a feature_surface path — while the docs/FEATURES.md row it owes had landed in an earlier commit on the branch. The range check was green and the per-commit check was red, which is exactly the failure mode #573 describes. The docs/FEATURES.md edit now rides in that commit, and it is a real one: the State cell said "refused by name", which overstated what the message gave back to a sub-divisor caller.

CI

windows-msvc-cpu and windows-msvc-vulkan are known-red on every pull request and have no main baseline (#584). sanitize-cpu and agent-record are NOT in that category any more — #873 and #904 are fixed on main — so a red in either on this head is new information and is not waived here in advance.

No GPU was used.

Owed

LTX-2.5: the res_2s DENOISING LOOP is unported, so TI2VidTwoStagesHQPipeline cannot be served — only its per-step arithmetic exists #921 — the res_2s denoising loop (samplers.py:208-447) is unported, so TI2VidTwoStagesHQPipeline cannot be served. What exists is one substep's SDE arithmetic (Ltx2Res2sStep/Ltx2Res2sSdeCoeff, gated). Absent are the phi/get_res2s_coefficients exponential integrator (res2s.py:1-60), the second transformer evaluation per step at sub_sigma = sqrt(sigma * sigma_next), the bong anchor refinement, and any Ltx2StepperKind enumerator to select it. No HQ recipe row is added here, so nothing can select it and nothing lands dead. Serving the HQ preset on the Euler loop would render a plausible clip that is quietly not HQ.
TI2VidTwoStagesPipeline as a distinct recipe row — covered by LTX-2.5 FULL PORT: close every refused arm, prove the prompted path on real weights, and lift the resolution ceiling #644.
Attributing the 60 GB decode loss, and the single-threaded decode throughput. Both need the GPU.
The lcm form of the divisor. max_spatial_downscale() takes the maximum where the quantity a request must survive is the least common multiple of the phase downscales. The two agree on every shipped recipe, whose downscales are 1 and 2, and part on a recipe with phases at 2 and 3: the max gives 96, a 96-wide request passes, and the downscale-2 phase then floors 48 onto one latent cell. This is narrowed and recorded rather than implemented, because no production entry point can reach the difference today and no test entering through one could gate it. The limit is stated in the header comment on Ltx2AssertResolution so the recipe row that adds a non-power-of-two downscale finds it, and under ## Owed in the spec.

Records

docs/FEATURES.md gains one appended row and nothing else: git diff origin/main -- docs/FEATURES.md is 1 0, one insertion and zero deletions, so no key origin/main wrote is disturbed. The spec had declared the file out of scope on the grounds that a refusal changes no feature surface; check-doc-checkpoint disagreed and was right, and the spec records the correction. Git auto-merged this keyed record without a textual conflict, which is not the same as a correct merge, so it was re-derived by key rather than trusted.

That row's widest cell is 212 characters against the 220 check-public-doc-tables.py refuses above — #964 pushed the LTX-2.5 DiT row's last cell to exactly 220, which passes because the comparison is strictly greater, and this row was written to fit under the cap on its own rather than by trimming somebody else's wording.

docs/USAGE.md differs from origin/main by exactly one hunk, and the four lines that hunk removes are byte-identical to the four this branch removed from the merge base. #964's two edits to the same file — the keyframe paragraph and the --offload-config row of the flags table — are untouched.

The READER ANCHORS list in ltx2_video.cpp was re-derived from the merged tree by the same walk test_ltx2_video performs, not assumed: the nine readers sit at 756 811 907 923 925 1003 1028 1133 1174, which is what the merge took from origin/main. This branch's additions to that file all fall after line 1174, so they move no reader, and the gate that would have caught it otherwise passes.

.agents/issue-index.md: two rows appended, zero removed. GitHub first reported this pull request CONFLICTING while git merge-tree reported the same pair clean, which is #883: the union driver .gitattributes declares for that path is a local driver and the forge does not run it. The index was the only file both sides touched, and both sides only appended. Merging origin/main here applied the driver and left the forge nothing to resolve. After the merge of c7cb59fbb the index was re-verified two ways: git diff origin/main -- .agents/issue-index.md shows exactly the two rows this branch appends and no removal, and a scan of every row key finds no duplicate, which is the shape a union merge of two relocations produces.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]

…ilently floor Nothing in this tree established what resolutions LTX-2.5 supports. `ltx2-gen` exposes `--width`, `--height` and `--frames`, and exposing a flag is not supporting a value. Reading the pinned oracle answers the three axes differently, and the difference is the whole finding. Upstream hard-validates width and height at the top of every pipeline `__call__` and raises: `assert_resolution` (ltx-pipelines utils/helpers.py:540-551), 64 for two-stage and 32 for one-stage, ten call sites including ti2vid_two_stages.py:184 and ti2vid_two_stages_hq.py:199. We check neither, and integer-divide instead (ltx2_video.cpp:1456-1463), so a 100x100 request renders 96x96 and returns success. Frames are the opposite answer: upstream floors an explicit num_frames exactly as we do (ltx_core/types.py:113) and validates it nowhere, because snap_frames_to_grid is reached only from the auto-duration path. So one axis owes a refusal and the other owes a documentation correction, and mirroring is what tells them apart. The spec also records what this row does NOT take. There is no maximum-resolution code cap to lift; the only geometry guard today is a lower bound. The real ceiling is host memory and decode throughput, it is already measured, and it is already unattributed - 448x256/25f loses ~59 GB in 24 s inside a decode whose own heap peak is 361.72 MiB. Attributing that needs the GPU, which this row must not use. The res_2s denoising loop is owed rather than approximated, because substituting Euler under the HQ preset renders something plausible that is quietly not HQ. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…ead of flooring it `vllm_video_generate` integer-divided `width` and `height` into the latent grid and checked only the lower bound, so an unsupported size rendered a clip at a size nobody asked for. Measured on the reduced fixture before the guard existed: a two-stage request of width 80 rendered 64x64 and returned success, and a one-stage request of width 100 rendered 96x64. Width 96 on the two-stage arm took the other route. Stage 1 floors 48 to one latent cell while stage 2 needs three, so the upsampler's shape check caught it and reported "the upsampled latent is 4x2x2x2 but phase 'refine' needs 4x2x2x3", a true statement about latents and no help to a caller who passed a width. The defect has two faces, a silent floor and an unreadable downstream throw, and one guard at the entry point closes both. `Ltx2AssertResolution` mirrors `assert_resolution` (ltx-pipelines utils/helpers.py:540-551), called where upstream calls it: the top of `__call__`, before any work is paid for. The divisor is DERIVED as the VAE spatial factor times the recipe's worst phase downscale, which reproduces upstream's 64 for a two-stage recipe and 32 for a one-stage one without restating either as a literal. Two subcases drive the same width 96 through both recipes and require opposite answers, so a hardcoded 64 fails the suite. Frames are deliberately left alone, and the asymmetry is upstream's: `resolve_num_frames` returns an explicit count verbatim and `VideoLatentShape.from_pixel_shape` (ltx_core/types.py:113) floors it exactly as this engine does, with `snap_frames_to_grid` reachable only from the auto-duration path this port does not serve. Adding a refusal there would diverge from the reference rather than mirror it, so `docs/USAGE.md` documents the rounding and publishes the envelope instead, with the measured 320x192/25f that completes set against the legal sizes that do not. Four mutations, each restored byte-for-byte. Deleting the production call site is RED, which is the reachability evidence. Hardcoding the divisor to 64 is RED on the one-stage subcases. Dropping the height axis is RED. Replacing the message's nearest-legal-size arithmetic with the request was GREEN on the first pass, so the suite gained the two assertions that make it RED: the suggested size is the user-facing contract, and a suggestion that is not itself legal sends the caller to another refusal. `docs/FEATURES.md` gains one row. The spec had declared it out of scope on the grounds that a refusal changes no feature surface; `check-doc-checkpoint` disagreed and was right, and the spec now records that correction. The "Temporal x2 ups gated, UNDRIVEN" cell is untouched. No upstream test is ported because there is none to port: Lightricks/LTX-2 at fd4ded7f contains zero test_*.py files anywhere in the repository. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

Brings in #903 so the geometry refusal is gated against current main rather than the base it branched from. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…e measurements The envelope table mixed a contract with observations of one box. The lead-in now says which rows are which, so a reader does not take 320x192 for a limit of the code. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…easurement changed Two decisions are only legible from the outcome section: the fixture sizes are measured because the obvious test value takes a different code path entirely and would have gated the wrong thing, and the one mutation that survived was against the refusal's suggested size rather than the check itself. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

GitHub reported the pull request CONFLICTING while `git merge-tree` reported the same pair clean, which is #883: the union driver `.gitattributes` declares for `.agents/issue-index.md` is a local driver, and the forge does not run it. The only file both sides touched is that index, and both sides only appended to it. Merging here applies the driver and leaves the forge nothing to resolve. Also brings in #842's fp8 W8A8 CPU path, so the gate reruns against it rather than against the base this row branched from. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

GitHub reported CONFLICTING a second time while `git merge-tree` reported the same pair clean, which is #883 again: `.agents/issue-index.md` carries a union driver that the forge does not run, so every main commit that appends a row re-conflicts every open branch that also appended one. `docs/FEATURES.md` and `docs/USAGE.md` now overlap as well. Both are keyed records, so neither is taken from the automatic three-way result on trust: each was re-verified after the merge by diffing against origin/main and confirming the only difference is this row's own scoped edit. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

`origin/main` moved to a318981 while this row was under review. #935 (LTX25-A2V-AUDIO-INPUT, c2019b0) inserted the audio-to-video block into `docs/USAGE.md` immediately above the `--frames`/width/height paragraph this row replaces with `#### The supported resolution envelope`, so the two edits conflict on content rather than on position. Resolved by keeping both: the A2V block byte-for-byte as `origin/main` wrote it, then this row's envelope section in place of the old paragraph. The resolved file is `origin/main`'s `docs/USAGE.md` with exactly one hunk applied, verified by diff. `docs/FEATURES.md` auto-merged, and it is a keyed record rather than an append-only log, so the three-way result was re-verified BY KEY: the merged file is `origin/main`'s with one appended row and every other row byte-identical. That row is 212 characters in its widest cell against `MAX_CELL_CHARS = 220`, which is the binding cap here (the row is 290 against `MAX_ROW_CHARS = 600`). FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…W6's own variable shadow (#965, #672) `windows-msvc-cpu` and `windows-msvc-vulkan` fail on every open pull request: server_main.cpp(1315,55): error C2220: the following warning is treated as an error server_main.cpp(1315,55): warning C4456: declaration of 'loaded' hides previous local declaration That is W6's own speech-attach block declaring `loaded` inside the scope of the text engine's `loaded` at `:1025`. It is the ONLY warning in the job, it has been on `main` since W6 landed, and it is fixed here by renaming the inner declaration. Nothing is suppressed and no detector is weakened. WHAT FOUND IT WAS THE MATCHED-ARM CHECK, NOT THE LABEL, and that is the part worth recording. Both jobs are habitually red and habitually attributed to #645 — which is the `M_PI` portability regression in three LTX2 sources: different file, different detector, different failure. A second cause sitting behind a known-red name is invisible for exactly as long as nobody reads the log. Three unrelated open pull requests that touch no speech surface — #956, #950 and #939 — fail with the identical C4456, which is what separates "pre-existing" from "mine". `main` carries no baseline because `windows-msvc-*` are PR-only (#584), so the failure presents to each author in turn as a red their own diff caused. Verified after the rename: 7 of 7 server ctest cases pass, `test_openai_api_server` is 62 cases / 733 assertions, and `vllm-server --speech-model <dir>` with no `--model` starts and serves for real: server: speech/music-only model (family=minimax-music3, 44100 Hz, text-only synthesis, family DETECTED); serving /v1/audio/speech server: listening on http://0.0.0.0:18923 (model 'minimax-music3') Issue: #965 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

The coordinator deleted this row's worktree while its agent was mid-session. The liveness check sampled `/proc/*/cwd`, and an agent thinking between Bash calls owns no process, so a live worktree read as dead. Everything staged was already in the object database; only the ref pointing at it was lost. This commit is that index, written back as a tree on top of the merge the agent had already committed. It is a RECOVERY, not new work: no line of the repair here was authored or reviewed by the coordinator, and every path but `docs/FEATURES.md` is byte-for-byte what `git write-tree` produced from the surviving index. `docs/FEATURES.md` is the one line added afterwards, and it is added here rather than in a later commit because `check-doc-checkpoint.py` walks PER COMMIT (#573): this change edits `src/vllm/model_executor/models/ltx2_pipeline.cpp`, so the feature surface it moves has to be recorded in the same commit or the gate is red on this SHA no matter what a later one says. The recorded move is real and not a formality — before this repair the refusal handed a sub-divisor caller `0x64`, a size the next guard rejects, so "refused by name" overstated what the message gave back. The State cell now says the refusal names the offending axis and a size the caller can actually pass. What it carries, per the agent's own report: the F1 axis-phrase repair with a both-axes subcase, the F2 sub-divisor repair naming the smallest legal size, the F3 recount (9 invocations, not ten or twenty-one), the F4 replacement of a false `snap_frames_to_grid` premise with a checkable one, and F5-F9. What it does NOT carry, and what the next implementer owes: the mutations were never run. M7 (swap the axis names) and the sub-divisor case are claimed by construction and unproven. The full gate never ran -- no ctest, no pass/fail line, no ENOSPC check. Treat every guarantee here as unverified until both are done. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

Main advanced to c7cb59f, whose #964 (LTX25-TOKEN-APPEND) edits the same three files this row does: `src/vllm/multimodal/ltx2_video.cpp`, `docs/USAGE.md` and `docs/FEATURES.md`. Git reported no textual conflict, which is not the same as a correct merge for a keyed record, so both documents were re-derived by key rather than accepted from the three-way result. `docs/USAGE.md` differs from `origin/main` by exactly one hunk, and the four lines it removes are byte-identical to the four this branch removed from the merge base. #964's two edits, the keyframe paragraph near line 501 and the `--offload-config` row of the flags table, survive untouched. `docs/FEATURES.md` differs from `origin/main` by one added row and no removed line, so no key #964 wrote is disturbed. That row's widest cell is 212 characters against the 220 the checker refuses above; #964 pushed the LTX-2.5 DiT row's last cell to exactly 220, which passes because the comparison is strictly greater. The READER ANCHORS list in `ltx2_video.cpp` needed no edit and is not assumed to: re-derived from the merged tree by the same walk `test_ltx2_video` performs, the nine readers sit at 756 811 907 923 925 1003 1028 1133 1174, which is the list the merge took from main. This branch's additions to that file all fall after line 1174, so they move no reader. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…the first, and it came from main (#968, #672) With #965's `C4456 'loaded' shadow` removed, `windows-msvc-cpu` and `windows-msvc-vulkan` failed again on this row's pull request — and on a different cause: include\vector(1461,29): error C2220: the following warning is treated as an error include\vector(1461,29): warning C4244: '=': conversion from 'const double' to 'float', possible loss of data raised from `src/vllm/multimodal/ltx2_video.cpp:203,214`, the two narrowing `positions.assign` calls that `c7cb59fbb` (#964, LTX25-TOKEN-APPEND) landed on `main` while this row was in flight. `StreamState::positions` and `Ltx2LatentState::positions` differ in element type; GCC and Clang narrow silently, MSVC diagnoses and the build treats it as an error. NOT FIXED HERE, deliberately. #964's own comment at `ltx2_video.cpp:129-132` reasons that "double -> float -> double reproduces the bits", so the narrowing is intentional and a silencing `static_cast` would be a claim about that reasoning rather than a formatting repair. It belongs to the lane that owns the round trip. Filed as #968 with the evidence rather than papered over. THE MATCHED ARM SPLITS EXACTLY ON THE MERGE BASE, which is what makes it inherited rather than mine. Grepping each `windows-msvc-cpu` job log for the warning: #966 and #951, both on `c7cb59fbb`, hit it twice each; #967, #956, #950, #939 and #938, all based before it, do not hit it at all. This row's diff touches zero LTX2 files. THE FINDING WORTH CARRYING is not either warning. It is that TWO INDEPENDENT CAUSES WERE STACKED BEHIND ONE HABITUALLY-RED JOB NAME, and the first hid the second — and that neither was #645, the `M_PI` regression both jobs are usually attributed to. A known-red list tells you a job is often red. It never tells you that today's red is the same one. Only reading the log does. Issue: #968 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]

…our conflicts by hand (#920) The branch was fifteen commits behind. Four files conflicted, and two of them are keyed records, so none of the four took an automatic three-way merge. The merged commit is named as a SHA rather than as `origin/main`, because `origin/main` moved between the first attempt at this merge and the second one. The first resolution was built against `c7cb59fbb`, and `e5351776c` (#939) landed while it was being written, rewriting 54 lines of the very `docs/USAGE.md` section this row edits. Merging the moving ref would have carried a resolution built against a tree that no longer existed. `include/vllm/multimodal/ltx2_video.h` and `src/vllm/multimodal/ltx2_video.cpp` conflicted with row LTX25-A2V-AUDIO-INPUT (#922), which added three per-generation extras beside this row's one. Both sides are additive and neither edits the other's text: the header takes main's `audio_path`, `audio_start_time` and `audio_max_duration` block followed by this row's `num_generated_keyframes` block, and diffing the resolved header against `e5351776c` gives 33 added lines and 0 removed. The `.cpp` extras check takes main's `known` predicate with this row's key added as a fifth disjunct and a fifth name in the message, rather than this branch's two-key `!=` chain, which main had already replaced. `docs/FEATURES.md` and `docs/USAGE.md` are keyed records, so this commit takes main's version of both BYTE-FOR-BYTE and re-applies nothing: `git diff e535177 -- docs/` is empty. The scoped edits belong to the repair commit that follows, because both were written against a tree where a supplied last-frame keyframe was refused, and #964 landed the seam that serves it. Re-applying them here would carry a claim this merge already knows to be false, and the FEATURES cell has no room for it either: the LTX-2.5 row's fourth column is 220 characters on main, which is `MAX_CELL_CHARS` exactly. The READER ANCHORS comment in `ltx2_video.cpp` was re-derived at this tree and is unchanged at `756 811 907 923 925 1003 1028 1133 1174`. Main's values carry, because every line this row adds sits below 1174, which is what the row's `## 7` predicted and what the gate now confirms rather than assumes. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

… heard, and the five keys upstream refuses that we dropped (#672, #953, #965) (#966) feat(MODEL-MUSIC-MUSIC3): a music-only server, an example that can be heard, and the five keys upstream refuses that we dropped (#672, #953) Developer directive: parity on what upstream supports — "we want to be a good reference" — usage docs for MiniMax-Music3, and in those docs the weights. Then, mid-flight: "we should allow to load only the music model" and "we need to have an e2e test working". FOLLOWING_AGENTS_PROTOCOL ## The upstream surface, enumerated rather than summarized SGLang-Omni `748a0b43` `models/minimax_music3/` and diffusers `c6da9936` `modular_pipelines/minimax_music3/`, read field by field and recorded with `file:line` in spec §10.1 so the next reader re-derives nothing. **Closed here:** the music-only server, the missing example, and five refusals. **Owed and named:** the non-`wav` response formats, request batching and `/v1/audio/speech/batch`, the 32 kHz delivery resample. **Permanently refused rather than owed:** streaming — neither upstream arm has it (`supports_streaming_vocoder=False`). **One place we are ahead of both arms:** `guidance_scale` is a real per-request control here, where diffusers freezes it at 1.7 into the guider component (`denoise.py:180`) and SGLang exposes it only as a serve-time knob. ## `--model` is optional when `--speech-model` is given Serving a 28.5 GB music model also forced loading an unrelated text model, and on this box the smallest text checkpoint is 35B — so the recipe this project documented was effectively unrunnable. Upstream's own is `sgl-omni serve --model MiniMaxAI/MiniMax-Music3`, no text tower anywhere. vllm-server --speech-model /path/to/minimax-music3 Third instance of a shape already in `server_main.cpp`: a pooling checkpoint serves `/v1/embeddings` alone, a Parakeet checkpoint serves `/v1/audio/transcriptions` alone. It mirrors vLLM's task-conditional registration (`api_server.py:255-265`). **Additive, and proved rather than argued.** The only case whose verdict changes is BOTH flags absent, which was an error and remains one with a message naming both options. The route table is gated in both directions over a real socket, because a handler-dispatch test cannot see route registration at all. ## The example the music family did not have `examples/minimax_music3_gen` — a thin client of `include/vllm.h` and nothing else, like `parakeet-transcribe` and `vllm-cli`. Hearing this model previously needed a running server plus a `curl`, or a C ABI caller nobody had written. ## Five keys upstream refuses by name were SILENT here (#953) `temperature`, `top_p`, `top_k`, `repetition_penalty` — refused upstream at `request_builders.py:14-19,109-114`, because this model's autoregressive stage has ONE sampler, a fixed top-50 draw (`encoders.py:48,94-103`). And `max_new_tokens`, upstream's LENGTH spelling in 25 Hz frames rather than seconds (`request_builders.py:56-68`), so a 250-frame request silently became the family's 60 s default. That is the #925 class exactly, in the same file that already carries #925's refusal one paragraph above. Fixed in flow. ## The e2e gate no longer reports a skip wearing a pass It read 5 cases / 5 passed with **`assertions: 0`** whenever the checkpoint was absent — the same shape that fooled this project on `test_qwen3_paged_engine`. Split into a checkpoint-free half that runs unconditionally in CI (request contract, both ceilings, the speech-only route table over a real socket with a stub synthesizer) and the env-gated half, whose HTTP case now drives the real engine over a real socket against the music-only server shape. A coverage-report case prints which arms ran, every run. The full arm was run: `POST /v1/audio/speech -> 200 audio/wav, 12332 bytes in 518 s wall`, 2 AR frames -> 6 latent frames -> 3072 samples/channel, 6144 int16 samples all non-zero, 0 clipped, 2818 of 3072 positions differing between L and R, and `/v1/completions` + `/v1/chat/completions` both 404 from the route table. `checkpoint_arms_run=5`. | arm | cases | assertions | |---|---|---| | `test_minimax_music3_e2e_real`, no env vars | 9 | 37 (was 5 / **0**) | | `test_minimax_music3_e2e_real`, checkpoint only | 9 | 86 | | `test_minimax_music3_e2e_real`, checkpoint + `VLLM_CPP_MUSIC3_DIT=1` | 9 | **582** | | `test_speech_api` | 6 | 67 | | `test_openai_api_server` | 62 | 733 | | `test_openai_conformance` | 23 | 252 | | `test_minimax_h3` (unchanged) | 79 | 57395 | | server flag ctest cases | 7 passed | | ## The weights are documented (porting-a-model.md §2.1) `docs/USAGE.md` gains component-by-component tables: the diffusers arm at `MiniMaxAI/MiniMax-Music3`@`fbdf52fbaaca799592917417eb05f1899f1255ec`, **28.5 GB resident** (28 517 617 303 B, measured) out of a 57.4 GB repository and why they differ; the native `.pth` arm we refuse and SGLang-Omni serves; the one implemented GGUF Q4_K artifact with its sha256; and the fourteen third-party quantized repositories in five formats, each marked refused. The revision is verified rather than copied — `condition_encoder/diffusion_pytorch_model.safetensors` on disk hashes to that revision's own LFS record. ## A sample a human can hear 2.0 s of 44100 Hz stereo from this engine in 3286 s of wall clock: RMS 0.03169, peak 0.97437 with 0 clipped samples, 84 073 of 88 064 positions differing between left and right. **Its samples are compared to nothing** — §5 withdrew the token gate — so it shows the pipeline runs, not that the music is right. It is not committed: `check-pr-size.py` classifies every path and none takes a `.wav` outside `tests/`, where a file compared to nothing would sit beside the goldens and imply it was one. ## The four asks, answered directly **1. Music-only server.** `vllm-server --speech-model <dir>` with NO `--model` starts and serves, observed live rather than inferred: server: speech/music-only model (family=minimax-music3, 44100 Hz, text-only synthesis, family DETECTED); serving /v1/audio/speech server: listening on http://0.0.0.0:18923 (model 'minimax-music3') `--model` alone and `--model` + `--speech-model` are **byte-identical in behaviour**. The whole change is one new early branch, `if (args.model_dir.empty())`, which loads the speech engine and `return 0`s before reaching a single line of the existing path; nothing downstream of it was touched. The only case whose verdict changes is BOTH flags absent, which was an error and remains one. Server suites: **7 of 7** ctest cases (4 pre-existing + 3 new — neither flag is still an error and now names both options; `--speech-model` alone reaches the speech LOAD; `--speech-family` alone still demands a checkpoint), `test_openai_api_server` **62 cases / 733 assertions** (+1 case / +24 assertions, the speech-only route table over a real socket), `test_openai_conformance` **23 / 252** unchanged. **2. e2e, three arms.** What a bare CI run executes unconditionally: the request contract on the exact body the real case posts, the near-miss and sampling refusals, the duration arithmetic including both ceilings, and the speech-only route table over a real socket with a stub synthesizer. What stays env-gated: everything needing the 28.5 GB checkpoint, plus the two 2.4B-DiT arms behind `VLLM_CPP_MUSIC3_DIT`. | arm | cases | assertions | checkpoint arms run | |---|---|---|---| | no env vars (what CI runs) | 9 | **37** | 0 — was 5 cases / **0 assertions** | | `VLLM_CPP_MUSIC3_CHECKPOINT` | 9 | **86** | 3 | | + `VLLM_CPP_MUSIC3_DIT=1` | 9 | **582** | 5 | **3. The five keys.** All five were **accepted and silently dropped**; all five are **now refused by name**. | key | upstream anchor | why it cannot be honoured | |---|---|---| | `temperature` | `request_builders.py:14-19,109-114` | the AR stage's only sampler is a fixed top-50 draw, `encoders.py:48,94-103` | | `top_p` | same | no nucleus branch exists | | `top_k` | same | `_AR_SAMPLING_TOP_K` is a module constant of 50 | | `repetition_penalty` | same | no penalty is applied anywhere in the loop | | `max_new_tokens` | `request_builders.py:56-68`, `constants.py:4-5` | upstream's LENGTH, in 25 Hz frames not seconds; the refusal names `audio_duration` and the /25 conversion | **4. The weights table** (`docs/USAGE.md`, "MiniMax-Music3: the exact weights"). It carries: repo **and** revision — `MiniMaxAI/MiniMax-Music3`@`fbdf52fbaaca799592917417eb05f1899f1255ec`, verified rather than copied, since `condition_encoder/diffusion_pytorch_model.safetensors` on disk hashes to `83179c5e…a202c2a4d`, that revision's own LFS record; the Q4_K artifact's sha256 `4c5d41b2…c70cbdd0` at revision `c36aaeed…` with its exact byte count; **28.5 GB resident (28 517 617 303 B, measured) versus 57.4 GB repository**, with the reason they differ; the refused native `.pth` arm (`qwen_7B/`, `flowmatching_vae.pth`, `dav.pth`) and that SGLang-Omni serves it; and all fourteen community quant repositories across five formats, each marked refused and each marked **third-party** rather than first-party. This is the first application of `.agents/porting-a-model.md` §2.1 (landing as #951). ## Two reds stacked behind one habitually-red job name `windows-msvc-cpu`/`windows-msvc-vulkan` are usually attributed to #645 (`M_PI` in three LTX2 sources). **Neither of the two causes here was #645**, and the first hid the second. **#965, fixed in flow.** `C4456: declaration of 'loaded' hides previous local declaration` at `server_main.cpp:1315` — W6's own speech-attach block declaring `loaded` inside the text engine's `loaded`. The only warning in the job, on `main` since W6 landed. Matched arm: #956, #950 and #939, none touching the speech surface, fail identically. Renamed; nothing suppressed. **#968, filed and NOT fixed here.** With the shadow gone the same jobs failed again on `C4244: conversion from 'const double' to 'float'`, raised inside MSVC's `<vector>` from `ltx2_video.cpp:203,214` — two narrowing `positions.assign` calls that `c7cb59fbb` (#964) landed on `main` while this row was in flight. **This branch touches zero LTX2 files.** The matched arm splits exactly on the merge base: #966 and #951 (on `c7cb59fbb`) hit it, #967/#956/ #950/#939/#938 (before it) do not. It is deliberately left to the LTX-2.5 lane — #964's own comment reasons that "double -> float -> double reproduces the bits", so a silencing cast is a claim about that reasoning rather than a formatting fix. **The finding, which outlives both:** a known-red list tells you a job is often red. It never tells you that today's red is the same one. Only reading the log does — and here it took two readings, because removing the first cause is what made the second visible.  ### The first of the two, in detail (#965) `windows-msvc-cpu`/`windows-msvc-vulkan` failed here, and they are **not** #645 (`M_PI` in three LTX2 sources). They were W6's own `C4456: declaration of 'loaded' hides previous local declaration` at `server_main.cpp:1315` — the only warning in the job, on `main` since W6 landed. The matched-arm check is what separated it from my diff: #956, #950 and #939, all touching no speech surface, fail identically. `main` has no baseline because `windows-msvc-*` are PR-only (#584), so it presents to each author in turn as their own red. Filed and fixed in flow by renaming the inner declaration; nothing suppressed. ## Mutations Four run, four fire: sampling refusal neutered (5 assertions red), `max_new_tokens` refusal neutered (2 red), `--model` made mandatory again (2 ctest cases red), generate routes registered unconditionally (3 cases / 6 assertions red in the api-server suite, 1 / 2 in the e2e suite). Sources restored and verified sha256-identical. Supersedes #954 (untrailered merge commits) and #963 (a `server_main.cpp` commit that owed `docs/USAGE.md` under the per-commit documentation checkpoint). Same tree, linear history, every commit green on `check-doc-checkpoint`, `check-commit-trailers` and `check-commit-style` locally before pushing. Every source file is byte-identical to the one built and gated. Issue: #672 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

…, refused by what is missing (#920) (#929) Upstream has two features called "keyframe", and only one of them touches the trained bias #658 landed. Supplied keyframes go through `VideoConditionByKeyframeIndex` and are **served** as of the token-append seam (#930, `c7cb59fbb`). **Generated keyframe slots** go through `VideoGeneratedKeyframeSlots` and have no request surface here: no key, no refusal, no test. Closes #920. Campaign #644. Spec `.agents/specs/ltx25-generated-keyframes.md`. ## The distinction, because it decides the whole row `extend_keyframes_mask` takes a `marked` argument, and over the pinned upstream `grep -rn "marked=True"` returns **exactly one hit**: `keyframe_slots.py:121`. Every other appending conditioning item passes `marked=False`, and `keyframe_cond.py:84-86` says why. So generated slots are the only user-facing feature that puts the marker on a token other than the target's own first latent frame. `KeyframeInterpolationPipeline` is **not** that feature. It builds only `VideoConditionByKeyframeIndex` items through `image_conditionings_by_adding_guiding_latent`, references neither `generated_keyframe` nor `keyframes_abs_pos` anywhere in its 362 lines, and is absent from the feature's own applies-to list at `docs/conditioning.md:47-51`. That was the working assumption when this row was dispatched, so the spec records the refutation rather than only the conclusion. ## What was wrong Asking for the feature got: ``` unknown per-generation extra 'num_generated_keyframes'. This family defines: image_crf ``` That message asserts the family does not define the key, which sends the reader looking for a typo instead of for the unported machinery. It is the same distinction `CheckUnservedExtras` was written for on the load side in #611, and `porting-a-model.md:81` requires the other answer: an unimplemented arm is refused with a message naming the missing piece. ## What the refusal names, and what this repair changed about it The first version of this refusal named **two** blockers, and by the time the branch reached review the first one was false in every particular. `c7cb59fbb` (row LTX25-TOKEN-APPEND, #930) landed twelve commits after this branch's merge base: it built the append seam, bound a `target_tokens` in the phase loop, turned the clear step from an explicit identity into a real trim, and **served** the LAST-frame supplied-keyframe arm the refusal had called "the SAME gap". Nothing went red. `ltx2_video.cpp` carries a running count of the refusals in this campaign whose stated reason turned out to be false or stale. It stands at six, and one of those six had a test written to assert the wrong reason BY NAME. This would have been seven, with its test. So the refusal now names **one** blocker, the **readback**, which is the half unique to this arm. The slots are the *output*: `apply_to` records a `GeneratedKeyframeLayout` that locates them exactly rather than assuming they trail, `clear_conditioning` extracts them into `generated_keyframes` *before* it trims, and each frame then has to be decoded as a standalone one-frame clip, because a K-frame causal decode blends slots that were never temporally adjacent. A port that grew the sequence and stopped would generate the slots and throw them away. Two reasons are **ruled out** with what ruled each one out, which is the shape the same message already used for `keyframes_abs_pos_embedding`: - **Not the token-append machinery.** #930 landed it and the last-frame arm runs on it. Two small pieces of that seam are still owed, and they are owed to *this* row rather than to #930: the `marked=true` branch of `Ltx2ExtendKeyframesMask` has no production caller, and `update_attention_mask` has no local counterpart because `Ltx2LatentState` deliberately carries no attention-mask field. - **Not `keyframes_abs_pos_embedding`.** #658 landed it and it applies on every render, because `_first_frame_keyframes_mask` marks the first latent frame unconditionally. ## The assertion that makes the next staleness go red Every assertion in the original case was on an **upstream** symbol name, and no change to *this* tree can move one, which is exactly why the suite stayed green through the commit that falsified the message. The reviewer measured that rather than arguing it: mutation **M7** replaced the local-cause sentence with a self-declared falsehood, left the upstream names alone, and the suite reported **18/18, exit 0**. The message now states its claims about this tree as two parsed lists, `DECLARED HERE:` and `ABSENT HERE:`, and the suite reads them **out of the thrown message** and re-derives each against `include/vllm/model_executor/models/ltx2_conditioning.h`. Three things keep that from being decorative: - it reads a **different file** from the one the claim lives in, because reading a span out of the file that makes it is the tautology #911 records; - it **strips comment lines** first, because that header's own comment names `VideoGeneratedKeyframeSlots` and an unstripped search inverts the ABSENT half — and the stripping is itself controlled, by requiring `struct Ltx2LatentState` to survive it; - both lists carry a **count floor**, because an empty list satisfies every `for each` and reports a pass over nothing. It fires in both directions. If the readback lands, `GeneratedKeyframe` appears in the header and ABSENT goes red, telling whoever landed it that this refusal is now false. ## `## Owed`, and a slice that was owned by nobody's spec `.agents/specs/ltx25-token-append.md` `## 8` assigns the unreached `Ltx2ExtendKeyframesMask(..., marked=true)` branch to this row by ID and issue. This row's `## Owed` did not list it. `AGENTS.md` `## Nothing lands dead` permits a staged unreached slice only while the **owning** row's spec lists it, so the permission rested on a bullet this row had never written. It is written now, with the branch's only driver named: `tests/vllm/models/test_ltx2_vae.cpp:2494` @ `e5351776c`, a unit case rather than reach. ## Two behaviours mirrored rather than collapsed "The key is present, so refuse" is one line shorter and wrong. An explicit `0` is upstream's own default and means off (`args.py:836`, `has_generated_keyframes`), so it renders. A negative count gets upstream's own reason from `evenly_spaced_keyframe_positions` rather than the unported-arm message, because a malformed request and an unported arm are different answers. The check runs before any arm is selected, so FP8, NVFP4 and bf16 cannot reach the unported readback by different routes. ## Reachability The test enters through the production entry point, `LoadVideoEngine` then `VideoEngine::Generate`, the chain `vllm_video_generate` takes. Proven by mutation, not asserted: see the table below. ## Mutations Four run against the repaired tree, each with three facts: `git diff --stat` after applying, whether it **BUILT** with the compile-error count beside it, and the **exit code**. Every run selected a non-zero case count (`1 | ... | 45 skipped`), because `-tc` with a comma selects nothing and prints SUCCESS. Each restored byte-for-byte, verified by `sha256sum -c` against a pre-mutation manifest, and rebuilt before the next measurement. | Mutation | diffstat | BUILT | exit / status | |---|---|---|---| | **M7 re-run** — the local-cause claims made false (`SERVED` -> `still REFUSED`, one name swapped across `DECLARED HERE:`/`ABSENT HERE:`), every upstream symbol left alone | 1 file, +3/-3 | yes, compile_err=0 | **1**, 1/1 case failed, 2 of 37 assertions, FAILURE | | **M7b** — the whole `LOCAL FACTS` clause deleted | 1 file, -8 | yes, compile_err=0 | **1**, 1/1 case failed, 1 of 18, FAILURE | | **reachability** — the production call site deleted (`grep -c` on the call returns 0), the definition marked `[[maybe_unused]]` so the mutation still builds | 1 file, +1/-2 | yes, compile_err=0 | **1**, 1/1 case failed, 13 of 18, FAILURE | | **zero-is-off** — `if (count == 0) return;` removed, so upstream's default refuses | 1 file, -1 | yes, compile_err=0 | **1**, 1/1 case failed, `31 \| 31 passed \| 0 failed`, FAILURE | M7's two failures name the stale claim in terms: *"the refusal claims ltx2_conditioning.h declares 'Ltx2GeneratedKeyframeLayout', and it does not. The message is stale about THIS tree"* and *"the refusal claims ltx2_conditioning.h has no 'Ltx2ExtendKeyframesMask', and it does."* Against the head this repairs, that same mutation was **green**. The last row is the reason the exit code is recorded rather than the assertion line: a thrown doctest case prints `0 failed` beside `Status: FAILURE!`. The five mutations the original head recorded are unchanged in intent and their guards are untouched by this repair; the two that share code with the repaired lines (reachability and zero-is-off) were re-run above rather than carried over. ## The merge, and the four conflicts `origin/main` moved between the first attempt at this merge and the second: the first resolution was built against `c7cb59fbb`, and `e5351776c` (#939) landed while it was being written, rewriting 54 lines of the very `docs/USAGE.md` section this row edits. The merge commit therefore names a **SHA** rather than the moving ref, and the resolution was redone against it. - `include/vllm/multimodal/ltx2_video.h` — additive against #922's three audio extras. Proof: `diff` against `e5351776c` is 33 added lines, 0 removed. - `src/vllm/multimodal/ltx2_video.cpp` — takes main's `known` predicate with this row's key as a fifth disjunct, not this branch's two-key `!=` chain, which main had already replaced. - `docs/FEATURES.md` and `docs/USAGE.md` are **keyed**, so the merge commit takes main byte-for-byte (`git diff e535177 -- docs/` empty) and the repair commit re-applies the scoped edit. FEATURES touches one line and USAGE is a pure insertion (17 added, 0 removed). `docs/FEATURES.md` had **no headroom**: the LTX-2.5 row's fourth column is 220 characters on main and `MAX_CELL_CHARS` is 220. The edit therefore trims rather than appends, and trims only wording this campaign owns — `IMAGE` becomes `IMG`, which column three already does, and `Speed PENDING` becomes `PENDING`, because that column's header is "Speed vs reference" and two sibling rows already write the bare word. No measurement and no host qualifier is touched. The cell lands at **218**. ## Anchors The `READER ANCHORS` comment in `ltx2_video.cpp` carries derived line numbers gated by `test_ltx2_video`. Re-derived at the merged tree — a **third** value was expected and none was needed: `756 811 907 923 925 1003 1028 1133 1174`, which is main's list unchanged, because every line this row adds sits below 1174. The branch's own pre-merge list, `690 745 841 857 859 929 954 1059 1100`, was correct for the pre-merge tree and is not the post-merge one; the merge took main's. The anchor case reports 1 case, 21 assertions, SUCCESS. Four `file:line` claims were wrong and are corrected here: | Claim | Was | Is | |---|---|---| | the `VideoGeneratedKeyframeSlots` class | `keyframe_slots.py:27-150` | `:27-174` — `27-150` ends at `apply_to` and excludes `_slot_positions` at `153-174`, which the same branch cited separately as class content | | `extract_generated_keyframes` | `tools.py:203-241` | `:203-230` — `233` begins an unrelated `AudioLatentTools` dataclass | | "`git grep -i generatedkeyframe` returns nothing" | 0 lines | true at the merge base, **7 lines in 4 files** at `e5351776c` (3 files outside `.agents/`) | | `.agents/issue-index.md` repeating blocker 1 | verbatim stale | rewritten before landing, since the index is append-only and cannot be corrected afterwards; every repo-local `path:NN` in it is SHA-anchored | ## Gate CPU-only. **The GPU was not used.** ``` CONFIGURE_EXIT=0 clean tree: build/ deleted first, so no incremental green BUILD_EXIT=0 ": error:" count 0 positive controls: 942 "Building CXX", 487 "Linking CXX" "No space left" 0, "BFD assertion" 0 ctest -N Total Tests: 485 CTEST_EXIT=0 100% tests passed, 0 tests failed out of 485 (2 skipped: test_modelopt_mixed_precision_checkpoint, test_voxtral_e2e) focused test_ltx2_video: 46 cases, 1094 assertions, SUCCESS, exit 0 scripts/agent-preflight.sh All gates green, exit 0 ``` **Load beside every number, because this box is shared.** The full build ran at 1-minute load 23 to 69 with other worktrees running `test_ltx2_video`, `test_parakeet_c`, `test_minimax_mu` and a `vllm-server`. `ctest` started at **11.56** and ended at **23.13**; the focused suite ran at 23.24 and ended at 21.63. Free disk was **14-15 GiB of 447** throughout, and no build or test line contains `No space left`. `check-doc-checkpoint` was run **per commit** (#573) and both pass, with the positive control armed: `--commit b5618b3` exits **1** with *"changed user_usage but did not update docs/USAGE.md"*. Nothing was known-red on this run. `test_cpu_x86_llamacpp_floor`, which failed with `NO_QUIET_WINDOW` on the previous head's gate, passed here. ## Answering #902 Reported on the issue. `Ltx2AdoptDeclaredDitParams` resolving to shapes is correct and already mirrors upstream; there is no live hole. Upstream resolves the same contradiction two ways at two layers by design: `LTXModel.supports_keyframes_abs_pos_embedding` reads the materialized tensor and returns False for a declare-true / carry-nothing checkpoint, while `DiffusionStage.supports_generated_keyframes` reads the declared flag only and would admit the request. Ours matches the first. If anything ours is safer, since upstream on that checkpoint would hand the forward a meta tensor; `enable_keyframes_abs_pos_embedding` exists for exactly that and has one hit repo-wide at the pin, its own definition, no caller. The residual on #902 is checkpoint availability, not code. ## Arms The refusal is resolved on the request ahead of arm selection, so bf16, FP8 and NVFP4 get one answer rather than one each. The arm itself — `GeneratedKeyframeLayout` readback, extraction before the trim, and the standalone single-frame decode — is **owed** under #920 and listed in the spec's `## Owed`, together with a production caller for `Ltx2ExtendKeyframesMask`'s `marked=true` branch. The token-append machinery is *not* among them: #930 landed it. GGUF k-quants are not applicable to this key and remain owed for LTX-2.5 as a whole under #644. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

…, and stop blaming the metadata (#923) (#938) Closes #923. Row `LTX25-IC-LORA`, spec [`.agents/specs/ltx25-ic-lora.md`](.agents/specs/ltx25-ic-lora.md). Upstream pin `Lightricks/LTX-2` @ `fd4ded7f2d88d3da713abcdd4ad41ecc4a9314ca`, verified at the local checkout with a clean tree before any anchor was read. Upstream `ICLoraPipeline` (`ltx-pipelines/ic_lora.py`) is video-to-video on the distilled model. This row builds the half of it that was actually missing — the adapter path — and leaves the other half refused, on the causes that genuinely remain. ## The reference refusal named a seam that had already landed This is the finding a fresh review returned FAIL on, and it is the most important thing in this pull request. The refusal originally blamed the IC-LoRA metadata. That was true, and this row closed it. The row then rewrote the refusal onto the **token-APPEND machinery**, which was accurate on 2026-08-15 and false on 2026-08-16: row `LTX25-TOKEN-APPEND` landed that seam in `c7cb59fbb` while this pull request was open, and the LAST-frame keyframe arm is served on it today. #964 left the wording byte-identical only because #938 was open and its cause was then still true. Both reasons this message had ever given were false at once. The determination was made **again, from the merged tree**, rather than inherited. The attention-strength wrapper is not in the way either: on the default arm upstream sets `attn_mask = None` at `conditioning_attention_strength >= 1.0` with no latent mask (`iclora_utils.py:159-160`) and applies `ConditioningItemAttentionStrengthWrapper` only `if attn_mask is not None` (`:168-169`), so #932 is not the blocker for the default case. **Two causes remain, and the refusal now names both.** 1. **The reference clip has no pixel path.** Upstream reads it at `height // scale` by `width // scale` (`iclora_utils.py:116-117`), refuses a target the factor does not divide (`:112-115`), keeps frame 0 then every Nth frame (`temporal_subsample`, `:87-89`, called at `:144`) and encodes the whole clip (`:145-148`). This engine's only pixel-to-latent route encodes exactly one frame at the phase's own resolution and refuses an encode returning more, and nothing anywhere reads `ref_video_dir`. 2. **The reference item is a STAGE-1 item and stage 2 must run UNFUSED.** `ic_lora.py:108` gives stage 1 `loras=tuple(loras)` and the reference conditioning (`:269-278`, `:377-402`); `:119` gives stage 2 `loras=()` and `:314-321` gives it `combined_image_conditionings` with no reference item. This engine holds ONE DiT, fused at load, that every phase runs. Serving the arm is filed as **#975** rather than done here. Piece 2 changes how the engine holds its weights, not how it conditions; a second resident DiT is ~21 B parameters, which is a memory decision this row cannot take on its own. ## The test could not have caught any of that, and now can The case asserted five substrings of the message. Two were **upstream** symbol names, present in the pinned checkout whatever this engine can do; three were literals the message declared about itself. None could go red when the engine changed. The reviewer measured it rather than arguing it: replacing the local-cause sentence with a self-declared falsehood while keeping all five substrings left `test_ltx2_video` at 44 cases / 914 assertions / SUCCESS. The case now **measures the engine first**. It renders with and without an appending conditioning item, reads `video_tokens` — the one trace field written inside the phase loop, so the only one that can observe what the loop does — and requires the grown count to exceed the plain one and both renders to return at the target frame count. Only then does it constrain the message, and the property is positional rather than lexical: every occurrence of a closed cause must sit after the `WHAT IS *NOT* THE REASON` marker, because recording a ruled-out cause is this message's own convention and has to stay possible. **R1 re-run on the repaired case**, at `f727cfd85`: `git diff --stat` 1 file, +13/-29; **BUILT** yes, `: error:` count 0; **exit 1**, 49 cases → 1 failed, 1140 assertions → 1 failed (the suite was 49 cases before the #929 merge added one). It fails on `REQUIRE(ruled_out != npos)` — the restored message has no ruled-out section at all — and would fail again on the positional check. ## The bf16 headline was only two thirds gated The aggregation dtype binds **three** roundings and only two had cases: `B * strength` (`fuse_loras.py:113`) and `deltas.add_(weight)` (`:67-68`). The matmul result's own `.to(dtype=dtype)` had none. **Measured:** widening only that one — keeping the f32 accumulator and adding the weight to it before the single store — left `test_ltx2_lora` at 13/13 and `test_ltx2_loader` at 31/31. The new case puts `acc = 1 + 2^-8` exactly on a bf16 tie and adds `w = 2^-9`, so the ported order stores 1.0 and an f32 accumulator stores 1.0078125 — one bf16 step apart in the STORED result, where the final rounding cannot absorb it. **Mutation:** `git diff --stat` 1 file, +4/-4; **BUILT** yes, `: error:` count 0; **exit 1**, `test_ltx2_lora` 14 cases → 1 failed, 2 assertions, reporting `16257 == 16256`. `test_ltx2_loader` stayed 31/31 under the same mutation, and the other 13 lora cases stayed green, which is the measurement that the hole was real. ## What lands * **The adapter reader** (`ltx2_lora.h` / `.cpp`): `.lora_A.weight` / `.lora_B.weight` pairs resolved onto the DiT contract through upstream's ComfyUI prefix strip (`sd_ops.py:135-137`), plus the file's whole `__metadata__`. * **The fusion**: `sum((B * strength) @ A)` added at load, mirroring `fuse_loras.py:99-116`. * **Every dtype arm from one hook**, placed immediately after `MaterializeDitTensor` because both quantized branches already `return vt::DType::kBF16`. On the streaming arm it runs before the device copy, so that arm's "one host buffer live at a time" invariant is unchanged. * **The surface**: `lora_path` / `lora_strength` load extras and `ltx2-gen --lora PATH [STRENGTH]`. No ABI change — both ride the existing parallel extras arrays. Load-time rather than per-request, because upstream takes the LoRAs as a `DiffusionStage.from_checkpoint` constructor argument (`ic_lora.py:104-114`). ## Two deliberate divergences, both argued rather than silent **We do not re-quantize.** Upstream's FP8 and NVFP4 rules dequantize, add, and re-quantize (`fp8_scaled_mm.py:167-189`, `nvfp4/fuse.py:13-50`) because they keep packed weights resident for their quantized kernels. This tree materializes bf16 on every arm and carries no FP8 or NVFP4 quantizer at all, so there is nothing to re-quantize into. Our fused weight skips upstream's lossy round trip and is slightly *more* precise on those two arms, at no extra bytes. **An adapter naming a module the contract lacks REFUSES.** Upstream skips it (`fuse_loras.py:135-137`) because its state dict is the whole model. Here the contract is a fixed enumerated set with unported modules already stripped, so a skip would absorb a misnamed key and an inapplicable one alike. ## `kLoraFusion` is retired, not reclassified It carried `DECLARED, NOT REQUESTABLE` — an assertion that no request field or load extra asks for LoRA fusion. One now does. #691 predicted this exact drift in its own words and records that the ledger test gates the message *text* rather than the property. The compiler caught it here, which is weaker than what #691 asks for and **does not close #691**. ## Reachability Proven on the rendered pixels, not on `last_conditioning()`: the conditioning trace is filled before the denoise loop, so it cannot see a fused weight, and a first attempt comparing it found every arm identical for that reason rather than because the LoRA did nothing. Production entry point: `vllm_video_engine_load` → `LoadVideoEngine` → `Ltx2VideoEngine::Load` → `Ltx2LoadDitFromSafetensors`. The test enters there with a `lora_path` load extra and compares rendered artifact bytes against an identical request with no adapter. ## Mutations | # | mutation | diff | BUILT | exit | result | |---|---|---|---|---|---| | M1 | accumulate the delta in f32 instead of bf16 | 1 file, +4/-4 | yes, no compile_err | 1 | 13 cases → 1 failed, 3 assertions | | M2 | **reachability**: delete `dit_options.loras.push_back` | 1 file, +2/-1 | yes, no compile_err | 1 | 44 cases → 5 failed, 8 assertions | | M3 | disable the unknown-target refusal (upstream's skip) | 1 file, +1/-1 | yes, no compile_err | 1 | 13 cases → 1 failed, 3 assertions | | M4 | ignore the adapter strength | 1 file, +1/-1 | yes, no compile_err | 1 | 13 cases → 1 failed, 6 assertions | | M5 | remove the zero-fusion refusal | 1 file, +1/-1 | **no** — `-Werror=unused-parameter` | NOT_RUN | establishes nothing; redone as M5b | | M5b | same, written to compile (`fused >= 0`) | 1 file, +1/-1 | yes, no compile_err | 1 | 31 cases → 1 failed, 2 assertions | | M6 | report the A factor under the B key | 1 file, +1/-1 | yes, no compile_err | 1 | 13 cases → 1 failed, 2 assertions | | **R1** | restore the pre-repair reference refusal verbatim | 1 file, +13/-29 | yes, no compile_err | 1 | 49 cases → 1 failed | | **F3** | widen ONLY the matmul-result rounding to f32 | 1 file, +4/-4 | yes, no compile_err | 1 | `test_ltx2_lora` 14 → 1 failed, 2 assertions; `test_ltx2_loader` 31/31 GREEN | R1 and F3 were run at `f727cfd85`, the commit before the #929 merge. Both carry to the pushed head: `git diff f727cfd e367026` is EMPTY over `ltx2_lora.cpp` and `test_ltx2_lora.cpp`, and over `ltx2_video.cpp` it changes only the refusal-counter comment and #929's own additions, leaving the reference `Fail(...)` string byte-identical. M1 to M6 were run before the review, at the counts the suites had then. Every mutation was restored byte-for-byte, proven by a clean `git status`. M5 is reported rather than dropped because a mutation that fails to build reads exactly like a passing test. ## On porting upstream's tests There are none to port. Measured at the pin with a positive control so a null result cannot be a wrong search term: `find -name 'test_*.py'` → 0, `find -type d -name 'test*'` → 0, `conftest.py` → 0, `grep -rl 'import pytest\|import unittest'` → 0, against `find -name '*.py'` → **280**. ## Records, and two merges resolved by key `origin/main` moved three times during this repair — #966, #939 and #929 — so the branch carries two merges. Both keyed records were resolved by taking main's version and reapplying this row's scoped edit. * **`docs/FEATURES.md`**: **186 of 188 keys byte-identical** to `origin/main`, the two that differ are the two this row owns, and no key is added or removed. Main's LTX-2.5 DiT cell had trimmed `IMAGE` to `IMG` and `Speed PENDING` to `PENDING` to fit `GENkf`; that trim is kept and this row's own wording carried the cost. The merged cell is **218 characters against the 220 `MAX_CELL_CHARS` limit**, and no measurement or host qualifier was touched. * **`docs/USAGE.md`**: main's served-last-frame-keyframe paragraph taken whole; the reference paragraph rewritten, because both sides' reasons for that refusal are now false, and the page records that rather than deleting it. * **`.agents/issue-index.md`**: the branch had appended a **second** `#930` row describing that blocker as open. #930 is closed, and main already carries the authoritative row, so the duplicate is dropped before it lands — the append-only rule protects rows that exist on `main`, and this one never did. The `#932` row's "blocked behind #930" is corrected, and **#975** is appended. `issue-index append-only` passes in preflight. * **`READER ANCHORS`**: both sides had rewritten the list, so neither survived the merge. Re-derived a third time with the same algorithm `test_ltx2_video` uses: `779 789 790 852 948 964 966 1044 1069 1174 1215`. * **The refusal counter stays at SIX.** #929 and this row both rewrote a refusal onto token-append, and `c7cb59fbb` falsified both while both were open in review. #929 landed leaving the counter at six, reasoning that a near-miss caught in review is not one of them; that reasoning applies here too. The comment records both near-misses rather than claiming a seventh, and a SIX and a SEVEN would have auto-merged into one of them silently. ## Gate ``` HEAD=e367026cd dirty_files=0 CONFIGURE_EXIT=0 BUILD_EXIT=0 ": error:" count=0 "No space left"=0 "BFD.*assertion"=0 control_no_space=1 control_bfd=1 control_error=1 free disk 42G ctest -N: Total Tests: 489 CTEST_N_EXIT=0 CTEST_EXIT=0 100% tests passed, 0 tests failed out of 489 loadavg 5.37 at start, 5.36 at end scripts/agent-preflight.sh: PREFLIGHT_EXIT=0 ``` The three `grep` counts carry **positive controls**, so a zero cannot be a broken instrument: the same patterns match their own sample strings and return 1. `scripts/check-doc-checkpoint.py` walks **per commit** (#573) and was run that way, armed with a positive control: `--commit b5618b3` exits **1** with the `user_usage`/`docs/USAGE.md` message, and both new commits plus the whole `origin/main..HEAD` range exit **0**. The gate on the pre-#929 tree was also green — 489/489, exit 0 — and is reported only to say that the #929 merge did not change the verdict. ## Owed * **#975** — serving the reference arms: the reference clip's own pixel path, and the stage split that gives stage 2 no adapter. Token-append is no longer part of it. * **#932** — the `conditioning_attention_strength < 1.0` / `conditioning_attention_mask` arm, and N-adapter fusion. The latter needs upstream's second rounding pattern (`addmm_` with `alpha`, `fuse_loras.py:115`), which this row refuses rather than guesses. GGUF k-quant LoRA fusion is **not applicable** rather than owed: the LTX-2.5 DiT ships FP8 and NVFP4, and no GGUF LTX DiT exists to fuse into. ## Not measured, and not implied No real-weights IC-LoRA fusion, and no render-quality or speed claim. **No GPU was used.** The fixture render moves 33 of 91169 artifact bytes, which is a reachability witness on a 2-layer reduced DiT and is not a quality result. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

mudler added 8 commits August 15, 2026 15:44

Merge origin/main into row/LTX25-RESOLUTION-ENVELOPE

4524aa6

Brings in #903 so the geometry refusal is gated against current main rather than the base it branched from. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]

localai-bot mentioned this pull request Aug 16, 2026

windows-msvc-cpu/vulkan are red on EVERY pull request: C4456 'loaded' shadow in server_main.cpp, and it is not #645 #965

Open

localai-bot mentioned this pull request Aug 16, 2026

feat(MODEL-MUSIC-MUSIC3): a music-only server, an example that can be heard, and the five keys upstream refuses that we dropped (#672, #953) #963

Closed

localai-bot merged commit e535177 into main Aug 16, 2026
19 of 25 checks passed

localai-bot mentioned this pull request Aug 16, 2026

feat(LTX25-GENERATED-KEYFRAMES): the second feature called "keyframe", refused by what is missing (#920) #929

Merged

localai-bot mentioned this pull request Aug 16, 2026

feat(LTX25-IC-LORA): read the IC-LoRA adapter, fuse it into every arm, and stop blaming the metadata (#923) #938

Merged

Navigation Menu

fix(#919): refuse a resolution the latent grid cannot represent, and publish the LTX-2.5 envelope - #939

fix(#919): refuse a resolution the latent grid cannot represent, and publish the LTX-2.5 envelope#939
localai-bot merged 10 commits into
mainfrom
row/LTX25-RESOLUTION-ENVELOPE

localai-bot commented Aug 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

localai-bot commented Aug 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The defect

Upstream raises, so we refuse

Frames are the opposite answer

The published envelope

Tests

Gate

CI

Owed

Records

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

localai-bot commented Aug 15, 2026 •

edited

Loading