feat(LTX25-GUIDED-VIDEO): the guided video denoiser, and the sentence in ltx2.h that was wrong (#1092) - #1102
Merged
Merged
Conversation
…r pipelines wait on the same seam (#1092) `Ltx2PhaseRecipe::video_guidance` is set by every recipe and read by nothing, so a `pipeline_kind = one_stage` render ignores the `cfg_scale = 3.0`, `stg_scale = 1.0`, `rescale_scale = 0.7` and `modality_scale = 3.0` its own recipe resolved. Upstream's `ti2vid_one_stage.py:221-226` @ `fd4ded7f` builds a `FactoryGuidedDenoiser` from exactly those. The spec commits before the implementation on the same pull request, which is this repository's default shape. It records three missing parts rather than one: the denoiser itself (`utils/denoisers.py:62-207`), the negative conditioning the video path never encodes, and the `SKIP_A2V_CROSS_ATTN` / `SKIP_V2A_CROSS_ATTN` perturbations without which `modality_scale = 3.0` — the default on every video row — cannot run. `ltx2.h:41-49` records those last two as not ported because "nothing upstream that this port serves constructs them"; that sentence was true for text-to-audio, which pins `modality_scale = 1.0`, and is false for every video pipeline. It also states what the row cannot claim: there is no oracle run behind any number in it. vLLM-Omni is UNPINNED (#633) and carries no LTX-2.5 recipe, and no LTX-2.5 checkpoint here has a recorded sha256 (#1048), so the guidance arithmetic is gated against upstream SOURCE and not upstream OUTPUT. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
… in ltx2.h that was wrong (#1092) The LTX-2.5 video denoise loop ran ONE unguided forward per step. Every recipe resolved a `video_guidance` that nothing read, so a `pipeline_kind = one_stage` render ignored `cfg_scale = 3.0`, `stg_scale = 1.0`, `rescale_scale = 0.7` and `modality_scale = 3.0` and denoised along a different trajectory than `ti2vid_one_stage.py:221-226` @ `fd4ded7f`, which builds a `FactoryGuidedDenoiser` from exactly those. `_guided_denoise` (`ltx-pipelines utils/denoisers.py:62-207`) is now ported, in its own translation unit because upstream has its own file. It assembles the four passes -- cond, uncond, perturbed, isolated-modality -- from the UNION of what the two guiders want, runs each through the caller's `X0Model`, and combines each modality with its own guider over the same splits. Four unported pipelines (`a2vid_two_stage.py:230`, `ti2vid_two_stages.py:248`, `ti2vid_two_stages_hq.py:271`, `keyframe_interpolation.py:232`) were each blocked on this one piece. ## The conversion lives in the caller's lambda, not in the seam `DiffusionStage` never hands the loop a velocity model: it hands `X0Model(builder.build(...))` (`utils/blocks.py:480-482`, forward at `model.py:590-604`). So the seam takes a callable and can only ever receive denoised tensors. Combining velocities and converting once afterwards is a different function wherever `rescale_scale != 0`, which is 0.7 on every video row of the params table. That defect shipped on the audio arm of this tree and is #1039. `post_process_latent` therefore stays OUT of the denoiser. Upstream applies it in the LOOP, to the guider's result (`utils/samplers.py:35`, `:484`). Applying it per arm would make every arm agree on exactly the conditioned tokens, which zeroes the guidance delta precisely where a keyframe or a reference clip is conditioning. ## The NOT-PORTED note in ltx2.h was wrong, not stale It refused `SKIP_A2V_CROSS_ATTN` and `SKIP_V2A_CROSS_ATTN` because "nothing upstream that this port serves constructs them -- STG is built from `stg_blocks` and reaches the SELF-attention types alone". STG does. The isolated-modality pass does not: `_guided_denoise` builds BOTH cross types with `blocks=None` whenever either guider has `modality_scale != 1.0` (`denoisers.py:121-137`), and every VIDEO row of the params table sets it to 3.0. The sentence was true of text-to-audio, which pins the field to 1.0 (`t2a_one_stage.py:200-202`), and it was written while text-to-audio was the only guided path here. Both directions are now `cross_attn_skip_all` booleans on `Ltx2DitPerturbation`, gating the A2V and V2A branches exactly as `transformer.py:335` and `:366` do. The same shape appears once more, and is corrected the same way: `negative_prompt` and the five `audio_*` guider knobs were refused on every non-t2a engine, on the same reading. `default_1_stage_arg_parser` carries the whole audio guider row beside the video one (`utils/args.py:1011-1075`) and `TI2VidOneStagePipeline` consumes both. That refusal is gone; the direction that survives is the one that refuses a knob describing a PICTURE on a pipeline that renders none. ## What a caller gains, and what is refused Seven per-generation extras mirror `default_1_stage_arg_parser`, reaching `ltx2-gen` and the C ABI. Every one is refused whole on a phase whose recipe sets `allow_guidance_override = false` -- the distilled and retake recipes, whose guidance is distilled into the weights. That field had never been read before. The unconditional forward needs a negative conditioning. With a tower it is the second half of the encode `GenerateAudioOnly` already performed and discarded; without one, `negative_prompt_embeds_path` and `negative_audio_prompt_embeds_path` are the negative half of the existing embeds fallback, and are recorded as a local adaptation rather than an upstream surface. With neither, a `cfg_scale` other than 1.0 is refused by name rather than served the positive context twice, which would leave the whole classifier-free term at exactly zero. Two more refusals exist because the alternative renders: an `stg_blocks` list that names no block this checkpoint has (upstream's membership test would perturb nothing, leaving `stg_scale * (cond - perturbed)` at zero), and the perturbed or isolated-modality pass on the device arm, where `Ltx2DitForwardDevice` takes no `perturbations` argument. ## What is unchanged `distilled_two_stage`, `dfr`, `retake` and `dmd2` keep the guider they always had, which is `Ltx2MultiModalGuiderParams`'s own default construction and is upstream's `_POSITIVE_ONLY_GUIDER` (`denoisers.py:25-28`). They issue one forward per step through the new seam and their goldens did not move. ## Evidence Spec [`.agents/specs/ltx25-guided-video.md`](.agents/specs/ltx25-guided-video.md), committed at `36510ec2d`, before this change. The gate asserts `x0 == latent - sigma*velocity` per token on ALL FOUR arms -- exact in x0 space, off by the whole sample in velocity space -- with the non-vacuity guard stated twice, once for a zero sample and once per arm for a zero velocity. It then replays `Ltx2MultiModalGuidance` over the recorded arms, and again over arms REBUILT FROM THE RAW VELOCITIES, which is the only check that can see something applied to every arm alike. It recovers the Euler step's input from the latent the sampler wrote. A seam-level control puts the two spaces 1.5e-07 apart at `rescale_scale = 0` and far apart at 0.7, with the modality term present, which the T2A control could not carry. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
`origin/main` advanced to `0bac476b7` while this row was in progress. Two record surfaces touched both sides and each was verified by hand rather than taken from the driver. `.agents/issue-index.md` is append-only under `merge=union`: the merged file's first 309 lines are BYTE-IDENTICAL to `origin/main`'s whole file, it carries exactly one extra line -- this row's `#1092` -- appended at the end, and its 292 row keys are unique. `docs/FEATURES.md` took main's MUSIC3 edit unchanged; this row's three LTX-2.5 rows sit elsewhere in the same table and neither side removed a row. No source file was touched by both sides. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…ity, and proving it took a conditioned render (#1092) The mutation pass on this row ran twelve mutations. Eleven went RED. The twelfth -- `post_process_latent` ADDED to each guidance arm, with the after-guider application left in place -- stayed green at 71 cases / 2145 assertions / exit 0, and this commit is what came of chasing it. The first reading was that the gate could not see it, so the gate grew a check that replays `Ltx2MultiModalGuidance` over arms REBUILT from `latent` and the four RAW velocities. That replay is independent of anything applied to every arm alike, which is exactly the shape of the mutation. It did not move the green either, and that was the clue. The second reading is that the mutation changes nothing. `post_process_latent` is `x*mask + clean*(1-mask)` (`utils/helpers.py:462-464`), so it can only touch a token whose denoise mask is 0. Such a token's per-token sigma is 0 (`timesteps_from_mask`, `:494-503`), so `X0Model` returns `latent - 0*v`, which is `latent`; and a conditioned token's `latent` IS its clean value, which the conditioner writes and the Euler step preserves. Every arm already equals what post-processing would write. ## The tempting next sentence is false, and that is the test this adds "So it does not matter where it is applied" does not follow. The guider's rescale (`guiders.py:268-271`) is a scalar over the WHOLE tensor, so it multiplies the conditioned tokens too: every guidance term is zero on a token where all four arms agree, leaving `pred = latent * factor` there. The after-guider `post_process_latent` is what pins those tokens back to `clean`. Remove it, or move it down into the denoiser, and the conditioned tokens leave the step scaled by a number nobody asked for, on a render that finishes. The unconditioned case could not see any of this: with every mask entry at 1, `post_process_latent` is a literal no-op and every placement of it passes. So this adds a CONDITIONED `one_stage` case, on the same guided configuration with an image conditioning, which asserts both halves -- that no arm was touched, and that the guider's result was, on exactly the mask-0 tokens -- behind a positive control that the post-process moved something at all. MEASURED: the MOVE mutation is RED against it (71 cases / 2133 assertions / exit 1, 0 compile errors, `1 file changed, 4 insertions(+), 3 deletions(-)`). The ADD mutation stays green, and the arm assertion now states why in an executable form rather than leaving a green unexplained. ## And one guard at the call rather than at the recipe `Ltx2DitForwardDevice` has no `perturbations` parameter, so the argument list silently drops one. The pre-loop refusal is a statement about the RECIPE; this adds the statement about the CALL, in the `X0Model` lambda itself, because the two are not the same check and a pass that reached the device arm with a perturbation would leave the STG and modality terms at exactly zero with nothing in the frames, the shapes or the counts to show for it. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
Bring the branch up to origin/main at 281e6a1 before re-gating the review repairs. Both sides touch docs/FEATURES.md and docs/USAGE.md, so a clean merge-tree is not a built one and the full gate is rerun on the merged tree. .agents/issue-index.md is a keyed append-only record, so the auto-merge result was discarded: origin/main's version was taken wholesale and this branch's single row re-appended. The 319-line prefix is byte-identical to origin/main, 302 rows carry no duplicate issue id, and this branch's row is byte-identical to the one it committed. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
… three mutations proved it (#1092) The fresh review returned CHANGES REQUESTED on `420f6b474`: one blocking finding and ten non-blocking ones. It reproduced the core of the row exactly — the seam, the x0 space, all four arms, the reachability, the gate — and independently confirmed the live defect the row fixes, so none of that is revisited here. B1, THE BLOCKING ONE. `Ltx2DitPerturbation`'s two cross booleans were gated together and not per direction. Three mutations of `ltx2_dit.cpp`, each built clean with zero compile errors and each exit status captured directly, were GREEN over the whole `test_ltx2_video` binary: the DiT ignoring `video_cross_attn_skip_all` (M12), ignoring `audio_cross_attn_skip_all` (M13), and SWAPPING which flag gates which direction (M15). Only ignoring BOTH (M14) was caught. A build that applies exactly one direction, or applies both to the wrong ones, renders — on the DEFAULT video arm, whose `modality_scale` is 3.0 — with the isolated-modality term half wrong. Two things made the shipped-path case blind to it. The end-to-end `MaxAbsDiffOf(video_first_modality, video_first_cond)` still fires with one direction applied, because the modality pass still differs from `cond`. And `Ltx2ConditioningTrace::video_modality_skipped_{a2v,v2a}` is assigned from the perturbation struct THE SEAM BUILT (`ltx2_denoisers.cpp:315-316`), so it records what was handed over and nothing about what the DiT did with it — while its message claimed the latter. That message is corrected rather than left to overstate. The repair is test-only. Separation comes from upstream's own predicates (`transformer.py:265-269`): `run_a2v` needs the VIDEO stream enabled and the audio stream merely PRESENT, and `run_v2a` the reverse, so a forward with `audio->enabled = false` runs A2V alone and one with `video->enabled = false` runs V2A alone. Each row asserts BOTH halves — the flag for that direction moves the written stream, and the flag for the other direction leaves it bit-identical — which is what makes the swap detectable and not only the omission. The `ltx2.h` disclosure that called M15 untestable is retired: its premise (both directions are off together on the shipped path) was true and its conclusion did not follow, because nothing obliges the separating test to use the shipped combination. M12, M13, M14 and M15 are now all RED. B3, the other finding with product code behind it. The EMPTY `stg_blocks` refusal is a real divergence and its recorded reason was wrong. Measured at Lightricks/LTX-2 `fd4ded7f`: `ltx-pipelines/docs/multimodal-guidance.md:13` documents "Set to `[]` to disable STG" in the same table and idiom as `stg_scale` -> 0.0; the field DEFAULTS to `[]` (`guiders.py:204`); the flags are `nargs="*"` (`args.py:979-985`, `:1039-1045`, `:1107-1113`) so the empty list has a CLI spelling, and `nargs="+"` was the one-character way to forbid it; `LTX_2_3_HQ_PARAMS` SHIPS it on both modalities (`constants.py:105`, `:113`); and a whole-tree search found no validation of `stg_blocks` at all, with the null results recorded. `blocks=None` means EVERY block and `blocks=[]` means NO block (`perturbations.py:26-33`), and `ApplyStgBlocksExtra` exists to keep PRESENT-and-empty distinct from ABSENT — which the refusal then made unreachable. Dropped in `ApplyGuidanceOverrides` and exempted in `check_reaches_a_block`; the out-of-range refusal stays, because that is a request disagreeing with the CHECKPOINT rather than a caller asking for nothing. Upstream does not skip the pass either (`do_perturbed_generation` reads `stg_scale` alone, `guiders.py:279-281`), so the new case asserts the pass RAN, perturbed no block, and returned `cond` bit for bit. One sub-claim of B3 is REJECTED on evidence. It argued that `audio_stg_blocks=""` is still accepted on `t2a_one_stage`. That path does return before `ApplyGuidanceOverrides`, and the request is still refused — by `ltx2_t2a.cpp:203-214`, which `git log -S` puts on `main` at `0b0b8900f` with #1032, not on this branch. So there is no asymmetry today; both arms refuse and both diverge from upstream. Fixing the video half creates one, which is why #1111 is filed, indexed, and listed under `## Owed` rather than left implied. It is not fixed in flow because it changes a landed row's gated behaviour and one of its cases. B6, the anchors, re-derived against `fd4ded7f` from the sentence making each claim rather than by reading text out of the cited span. `_guided_denoise` is 61-211 and not 62-207; `enabled = not skip` is at 158 and 168, where 151 and 161 are the `= None` initializers; the V2A guard is 367 and 366 is blank; the batched config is built at 182-187, where 172-176 is a comment plus the per-sample replication; the partial blend is 572-573; the one `PromptEncoder` call is 166-174; `default_1_stage_arg_parser` is 930-1067 with its guider flags at 947-1066; `cross_attn_skip_all` is DECLARED at `transformer_args.py:70` and 118 is a call site; `modality_scale = 3.0` is at `constants.py:54, :64` with `_PARAMS_SINCE_VERSION` at 130-133, and the cited 40-80 covers neither; `CFGGuider` and `STGGuider` are 11-27 and 56-74; the `perturbations` ARGUMENT is `model.py:493` and 492 is the `def`. No gate protects a spec anchor (#632), so the 43 replacements were applied by a script that asserts the expected hit count per edit and refuses the whole run on a mismatch; two were caught that way and re-derived. The rest. B2: `docs/FEATURES.md` still called T2A "the only GUIDED arm", which this row's own new row two lines below made false — corrected inside the existing cell at 202 of 220 chars, with the page's prose-paragraph count unchanged at 21 of 21, because adding a paragraph there re-reds `main` for the whole repo (#1055). B4: `INFO("arm = " << arm.name)` printed `arm = 1`, doctest stringifying a `const char*` through its bool overload, so a single-arm regression could not be attributed from the output — wrapped in `std::string` at all three sites in the file. B5: the rescale control's modality claim is structurally true and numerically inert, and the case now MEASURES that (`4.054e-01` at `modality_scale` 3.0 against `4.118e-01` at 1.0) instead of implying coverage it does not provide; the modality arm's gate is the per-arm invariant, which M4 turns red. B7: "the seam cannot be handed a velocity" is caller discipline and not a type guarantee, since `Ltx2X0Outputs` carries the velocity beside the prediction — the claim is restated and the code left alone, because dropping the velocity would delete what the invariant is checked against. B10: the new `docs/USAGE.md` section gains the `/v1/videos` caveat its two siblings carry, a flag-to-extra table with the raw key spellings an ABI caller needs, and the empty-list behaviour B3 decided. B8: `origin/main` is merged in and the gate rerun on the merged tree. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
Second merge on this branch: origin/main advanced to 8fa405b (#595, the doc-checkpoint feature trigger) while the review repairs were being gated, so the gate reruns on the merged tree rather than on a tree that only merges cleanly. .agents/issue-index.md is a keyed append-only record, so the auto-merge result was discarded again: origin/main's version was taken wholesale and this branch's two rows re-appended in the order it committed them. The 320-line prefix is byte-identical to origin/main, and the file carries 304 rows with 304 distinct issue ids. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…te measures (#1092) `docs/FEATURES.md`'s evidence cell for the cross-attention perturbations read "asserts the isolated-modality pass reached the DiT with both directions off and moved the tensor", which is exactly the claim the review found overstated: that assertion is satisfied by a build applying one direction, or the wrong two. The cell now names what the gate actually does — each direction alone, on a forward where one stream is PRESENT but DISABLED so only that direction runs, and with the other flag required to leave the stream bit-identical, which is the half that detects a swap. Cell measured at 218 of 220 chars and the page's prose-paragraph count is unchanged at 21 of 21, so nothing else on the page has to move. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
localai-bot
marked this pull request as ready for review
August 17, 2026 08:58
Third merge on this branch: origin/main advanced to 9b3317c while the repair was being gated. The incoming change is `qwen3_5_weights.cpp` and touches nothing this row owns, but the trailer and commit-style gates decline to report at all on a branch that is behind, so the merge is what makes them say something about this tree. .agents/issue-index.md is a keyed append-only record, so origin/main's version was taken wholesale again and this branch's two rows re-appended in commit order. The prefix is byte-identical to origin/main and the file carries 305 rows with 305 distinct issue ids. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The LTX-2.5 video denoise loop ran ONE unguided forward per step. Every recipe
resolved a
video_guidancethat nothing read, so apipeline_kind = one_stagerender ignored
cfg_scale = 3.0,stg_scale = 1.0,rescale_scale = 0.7andmodality_scale = 3.0and denoised along a different trajectory thanti2vid_one_stage.py:221-226@fd4ded7f, which builds aFactoryGuidedDenoiserfrom exactly those.Issue #1092. Spec
.agents/specs/ltx25-guided-video.md,committed at
36510ec2d, before any code._guided_denoise(ltx-pipelines utils/denoisers.py:61-211) is now ported, inits own translation unit because upstream has its own file. It assembles the four
passes -- cond, uncond, perturbed, isolated-modality -- from the UNION of what
the two guiders want, runs each through the caller's
X0Model, and combines eachmodality with its own guider over the same splits. Four unported pipelines
(
a2vid_two_stage.py:230,ti2vid_two_stages.py:248,ti2vid_two_stages_hq.py:271,keyframe_interpolation.py:232) were each blockedon this one piece; none is any more.
The conversion lives in the caller's lambda, not in the seam
DiffusionStagenever hands the loop a velocity model: it handsX0Model(builder.build(...))(utils/blocks.py:480-482, forward atmodel.py:590-604). The seam therefore takes a callable and can only everreceive denoised tensors. Combining velocities and converting once afterwards is
a different function wherever
rescale_scale != 0, which is 0.7 on every videorow of the params table. That defect shipped on the audio arm of this tree and is
#1039.
post_process_latentstays OUT of the denoiser. Upstream applies it in the LOOP,to the guider's result (
utils/samplers.py:35,:484), and the difference isnot cosmetic: the rescale is a scalar over the whole tensor, so it multiplies the
conditioned tokens too, and the after-guider application is what pins them back
to
clean.The NOT-PORTED note in ltx2.h was wrong, not stale
It refused
SKIP_A2V_CROSS_ATTNandSKIP_V2A_CROSS_ATTNbecause "nothingupstream that this port serves constructs them -- STG is built from
stg_blocksand reaches the SELF-attention types alone". STG does. The isolated-modality pass
does not:
_guided_denoisebuilds BOTH cross types withblocks=Nonewhenevereither guider has
modality_scale != 1.0(denoisers.py:125-138), and everyVIDEO row of the params table sets it to 3.0. The sentence was true of
text-to-audio, which pins the field to 1.0 (
t2a_one_stage.py:202), and itwas written while text-to-audio was the only guided path here. Both directions
are now
cross_attn_skip_allbooleans onLtx2DitPerturbation, gating the A2Vand V2A branches exactly as
transformer.py:335and:367do.The same shape appears once more and is corrected the same way.
negative_promptand the five
audio_*guider knobs were refused on every non-t2a engine, on thesame reading of upstream.
default_1_stage_arg_parsercarries the whole audioguider row beside the video one (
utils/args.py:947-1066, the audio row opening at:1008) andTI2VidOneStagePipelineconsumes both (ti2vid_one_stage.py:211-218). Thatrefusal is gone; the direction that survives refuses a knob describing a PICTURE
on a pipeline that renders none. The case that asserted the old behaviour is
rewritten rather than deleted, because "this used to be refused" is what a later
reader needs.
What a caller gains, and what is refused
Seven per-generation extras mirror
default_1_stage_arg_parser, reachingltx2-genand the C ABI:--video-cfg-guidance-scale,--video-stg-guidance-scale,--video-rescale-scale,--video-skip-step,--video-stg-blocks,--a2v-guidance-scale,--v2a-guidance-scale. Every one isrefused whole on a phase whose recipe sets
allow_guidance_override = false--the distilled and retake recipes, whose guidance is distilled into the weights.
That field had never been read.
The unconditional forward needs a negative conditioning. With a tower it is the
second half of the encode
GenerateAudioOnlyalready performed and discarded;without one,
negative_prompt_embeds_pathandnegative_audio_prompt_embeds_pathare the negative half of the existing embedsfallback. That pair is a LOCAL ADAPTATION and is recorded as one in the spec:
upstream has no embeds surface at all. With neither, a
cfg_scaleother than 1.0is refused by name rather than served the positive context twice, which would
leave the whole classifier-free term at exactly zero.
Two further refusals exist because the alternative renders. An
stg_blockslistnaming no block this checkpoint has would perturb nothing under upstream's
membership test, leaving
stg_scale * (cond - perturbed)at zero. And theperturbed or isolated-modality pass on the device arm cannot run at all:
Ltx2DitForwardDevicetakes noperturbationsargument, so it is refused byname rather than served an unperturbed forward. That arm is OWED and the spec
lists it under
## Owed.What is unchanged
distilled_two_stage,dfr,retakeanddmd2keep the guider they alwayshad, which is
Ltx2MultiModalGuiderParams's own default construction and isupstream's
_POSITIVE_ONLY_GUIDER(denoisers.py:25-28). They issue one forwardper step through the new seam, and no golden in the suite moved.
Reachability
Entry point:
vllm_video_generate->VideoEngine::Generateon an engine loadedwith
pipeline_kind = one_stage, a documented value of a documented load extrathat needs no other flag. Every gate case enters there; nothing constructs a
guider, a DiT, a modality or a perturbation by hand.
Deleting the production call site -- the
Ltx2GuidedDenoiseline in the phaseloop, replaced by the single unguided forward this change removes -- turns the
suite RED at 3 failed cases and 13 failed assertions (M11 below).
The gate
test_ltx2_video: 71 cases, 2145 assertions, exit 0.It asserts
x0 == latent - sigma*velocityper token on ALL FOUR arms, with thePER-TOKEN sigma rather than the schedule scalar: exact in x0 space, off by the
whole sample in velocity space. Non-vacuity is stated twice, once for a zero
sample and once per arm for a zero velocity. Each arm is also asserted to DIFFER
from the conditional arm, so a pass whose context or perturbation never reached
the forward fails rather than passing as a perfectly converted copy.
It then replays
Ltx2MultiModalGuidanceover the recorded arms, and again overarms REBUILT FROM THE RAW VELOCITIES, and recovers the Euler step's input from
the latent the sampler wrote. A seam-level control puts the two spaces apart only
at a non-zero rescale, with the modality term present, which the T2A control
could not carry.
A second case runs the same guided configuration WITH an image conditioning,
because
post_process_latentis a literal no-op without one.Mutations
Twelve, each reporting three facts, because two of them are how a mutation lies:
that it applied (
git diff --stat), that it BUILT (compile-error count), and theexit code captured directly rather than after a pipe.
ToDenoisedBELOW the step-0 recordToDenoisedABOVE the step-0 recordpost_process_latentADDED per armpost_process_latentMOVED per armLtx2GuidedDenoisecall site deletedEvery restore was verified with a scoped
git diff --statreporting clean, andevery restored file had its mtime bumped before the rebuild, because a restored
file with an older mtime lets ninja skip and the NEXT measurement runs the
PREVIOUS mutation's binary.
M10a is a no-op, and saying so took a measurement.
post_process_latentisx*mask + clean*(1-mask), so it can only touch a mask-0 token; such a token'sper-token sigma is 0, so
X0Modelreturnslatent - 0*v, which islatent; anda conditioned token's
latentIS its clean value. Every arm already equals whatpost-processing would write. The first response to the green was to strengthen
the gate -- that is where the rebuilt-from-velocities replay came from -- and
when that did not move it either, the conditioned case was written to state the
identity in an assertion. M10b, the placement that actually changes the render,
is RED against it.
Gate numbers
420f6b4741ef8b5faae0874ccfb2d622d7c4a7d6, remote-verified withgit ls-remoteCONFIGURE_EXITBUILD_EXIT: error:count 0ctest -NCTEST_EXITtest_modelopt_mixed_precision_checkpoint,test_voxtral_e2e)No space left/BFDPassedmatches 496 linesWhat this does NOT claim
No number here comes from a running oracle. vLLM-Omni is UNPINNED
(#633) and carries no LTX-2.5
recipe; no LTX-2.5 checkpoint on this host has a recorded sha256
(#1048). The guidance is gated
against upstream SOURCE at
fd4ded7f, not upstream OUTPUT, and the spec'shonesty statement says so.
#1049 is PARTLY retired:
Ltx2BatchedPerturbationConfignow has a product caller.Ltx2Guidance,Ltx2CfgDeltaandLtx2StgDeltastay dead, and the spec argues why routingthrough them would mean inventing a kind dispatch upstream does not have rather
than closing a wiring gap.
The fresh review, and what it moved
The review returned CHANGES REQUESTED on
420f6b474: one blocking finding andten non-blocking. It reproduced the core exactly -- the seam, the x0 space, all
four arms, the reachability, the gate -- and independently confirmed the live
defect: at the merge base
video_guidancehad exactly two hits, a declarationand a write with no reader, against a positive control where
audio_guidancefinds its T2A consumer. None of that is revisited below.
B1, the blocking one: the two cross booleans were gated together
Ltx2DitPerturbation's two flags reachedLtx2BlockArgsand the gate could nottell which of them the DiT applied. Three mutations were GREEN over the whole
binary: the DiT ignoring
video_cross_attn_skip_all(M12), ignoringaudio_cross_attn_skip_all(M13), and swapping which flag gates whichdirection (M15). Only ignoring both (M14) was caught. A build that applies one
direction, or applies both to the wrong ones, renders -- on the DEFAULT video
arm, whose
modality_scaleis 3.0 -- with the isolated-modality term half wrong.Two things made the shipped-path case blind to it. The end-to-end
MaxAbsDiffOf(video_first_modality, video_first_cond)still fires with onedirection applied, because the modality pass still differs from
cond. AndLtx2ConditioningTrace::video_modality_skipped_{a2v,v2a}is assigned from theperturbation struct the seam built (
ltx2_denoisers.cpp:315-316), so itrecords what was handed over and nothing about what the DiT did with it -- while
its message claimed the latter. That message is corrected rather than left to
overstate what it measures.
The repair is test-only, and the separation comes from upstream's own
predicates.
run_a2vneeds the VIDEO stream enabled and the audio streammerely PRESENT;
run_v2aneeds the reverse (transformer.py:265-269). So aforward with
audio->enabled = falseruns A2V alone, and one withvideo->enabled = falseruns V2A alone -- the configurationltx2.halreadydocuments as rendering rather than failing. Each row asserts BOTH halves: the
flag for that direction MOVES the stream it writes, and the flag for the other
direction leaves it BIT-IDENTICAL. The second half is what detects the swap.
Doing this on a both-enabled forward is impossible on this fixture, because block
1's V2A reads what block 0's A2V wrote.
ltx2.h's disclosure that called M15 untestable is retired. Its premise wastrue -- both directions are off together on the shipped path -- and its
conclusion did not follow, because nothing obliges the separating test to use the
shipped combination.
ltx2_dit.cppgit diff --statvideo_cross_attn_skip_allmoved(a2v_off.video, base.video)audio_cross_attn_skip_allmoved(v2a_off.audio, base.audio)mod != condEvery run was the WHOLE binary rather than a
--test-casefilter, becauseseveral case names here contain commas and doctest splits
-tcon them; atruncated filter matches zero cases and prints
SUCCESS!with exit 0. Each runis recorded with its case and assertion counts so a zero-count run cannot pass
for a green one: baseline 72 cases / 2182 assertions / exit 0, and each
mutation ran the same 72 and 2182 with 1, 1, 2 and 1 cases failing respectively.
Every restore was byte-verified and mtime-bumped before the rebuild.
B3, the other finding with product code behind it
The empty-
stg_blocksrefusal is a real divergence and the recorded reason waswrong. Measured at
fd4ded7f:[]to disable STG", in the same table and idiom asstg_scale-> 0.0 andcfg_scale-> 1.0ltx-pipelines/docs/multimodal-guidance.md:13MultiModalGuiderParams.stg_blocksDEFAULTS to[]guiders.py:204nargs="*", so[]has a CLI spelling;nargs="+"was the one-character way to forbid itargs.py:979-985,:1039-1045,:1107-1113LTX_2_3_HQ_PARAMSSHIPSstg_blocks=[]on both modalitiesconstants.py:105,:113stg_blocksanywhere in that treeblocks=Nonemeans EVERY block andblocks=[]means NO block(
perturbations.py:26-33), andApplyStgBlocksExtraexists to keepPRESENT-and-empty distinct from ABSENT -- which the refusal then made
unreachable. Dropped in
ApplyGuidanceOverrides, exempted incheck_reaches_a_block. The out-of-range refusal stays, because that is arequest disagreeing with the CHECKPOINT rather than a caller asking for nothing,
and upstream never meets it (48-block checkpoints only).
Upstream does not skip the pass either:
do_perturbed_generationreadsstg_scalealone (guiders.py:279-281). The new case asserts the pass RAN,perturbed no block, and returned
condbit for bit, with a named-blockcontrol beside it so the case is about emptiness rather than about the extra
being read at all.
One sub-claim is REJECTED on evidence. The finding argued that
audio_stg_blocks=""is still accepted ont2a_one_stagebecause that pathreturns before
ApplyGuidanceOverrides. It does return there, and the request isstill refused -- by
ltx2_t2a.cpp:203-214, which builds the block mask and failswhen no bit is set, and which
git log -Sputs onmainat0b0b8900fwith#1032, not on this branch. So
there is no asymmetry today: both arms refuse and both diverge from upstream.
Fixing the video half creates one, which is why
#1111 is filed, indexed, and
listed under
## Owed. It is not fixed in flow because it changes a landed row'sgated behaviour and one of its cases.
B6, the anchors
Re-derived against
fd4ded7ffrom the sentence making each claim, never byreading text out of the cited span.
_guided_denoiseis 61-211, not 62-207.enabled = not skipis at 158, 168; 151 and 161 are the= Noneinitializers. The V2A guard is 367; 366 is blank. The batched config is built
at 182-187; 172-176 is a comment plus the per-sample replication at
:175.The partial blend is 572-573. The one
PromptEncodercall is 166-174.default_1_stage_arg_parseris 930-1067 with its guider flags at947-1066. The two
--*-stg-blocksflags open at 979-985 and1039-1045.
cross_attn_skip_allis DECLARED attransformer_args.py:70;118 is a call site.
modality_scale = 3.0is atconstants.py:54, :64and_PARAMS_SINCE_VERSIONat 130-133, so the cited 40-80 covered neither.CFGGuiderandSTGGuiderare 11-27 and 56-74. TheperturbationsARGUMENT is
model.py:493; 492 is thedef.No gate protects a spec anchor
(#632), so the 43 replacements
were applied by a script that asserts the expected hit count per edit and refuses
the whole run on a mismatch. Two were caught that way and re-derived.
The rest
docs/FEATURES.mdstill called T2A "the only GUIDED arm", made false by this PR's own row two lines below. Corrected inside the existing cell at 202 of 220 chars; the page's prose-paragraph count is unchanged at 21 of 21, because adding a paragraph there re-redsmainfor the whole repo (#1055).check-public-doc-tables.pygreen.INFO("arm = " << arm.name)printedarm = 1, doctest stringifying aconst char*through its bool overload, so M1-M4 produced byte-identical failure context. Wrapped instd::stringat all three sites in the file.4.054e-01atmodality_scale3.0 against4.118e-01at 1.0, asserted to agree within a factor of two. Restated in the case title, the comment and spec 7.2, so a later reader cannot lean on this control for modality coverage -- the modality arm's gate is the per-arm invariant, which M4 turns red.Ltx2X0Outputscarries the velocity beside the prediction, so a lambda that swaps them compiles and renders. The claim is restated and the code left alone, because dropping the velocity would delete what the invariant is checked against. M1-M4 are the real gate.origin/mainmerged in twice (it moved during the repair) and the gate rerun on the merged tree. Spec 1 now carries #1093-#1096 from281e6a120..agents/issue-index.mdwas rebuilt both times by takingorigin/mainwholesale and re-appending this branch's rows: 320-line prefix byte-identical, 304 rows, 304 distinct ids.docs/USAGE.mdsection gains the/v1/videoscaveat its two siblings carry, a flag-to-extra table with the raw key spellings an ABI caller needs, the audio row's spellings, the load-extra status of the two negative-embeds keys, and the empty-list behaviour B3 decided. The rows are placed in that section rather than in the retake-scoped table at:3128, where they would be filed under the wrong pipeline.Gate numbers, on the merged head
8e0f19650dffa5b4daff6ec16ca9e27a49dc8508, remote-verified withgit ls-remoteCONFIGURE_EXITBUILD_EXIT: error:count 0ctest -NCTEST_EXITtest_modelopt_mixed_precision_checkpoint,test_voxtral_e2e)09aee8cc2, under load 134-168 from other agents on the box,test_serve_low_tools(#428) andtest_engine_core_proc(#1052) failed and both Passed on a serial re-run at load 71, exit 0. Both are green in the run abovetest_ltx2_videoNo space left/BFDLinkinglines in the build log, 497Passedlines in the ctest logcheck-test-registrationfailed with "Cannot open file for write" plus CMake's misreported "Inappropriate ioctl for device" -- an ENOSPC wearing a verdict about the code, confirmed byddwriting 22 of 64 MB and by the same gate passing once space returnedFOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]