diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 53a7287b8..e9b99ac00 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -252,4 +252,6 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#930](https://github.com/mudler/vllm.cpp/issues/930) | `LTX25-TOKEN-APPEND` | LTX-2.5's phase loop is fixed at one `Ltx2VideoTokenCount(vshape, 1)`, and that single limitation blocks THREE conditioning arms, not the two #930 was written against: reference video, the LAST-frame keyframe, and generated keyframe slots ([#920](https://github.com/mudler/vllm.cpp/issues/920)). Row `LTX25-TOKEN-APPEND` (spec [`ltx25-token-append.md`](specs/ltx25-token-append.md)) ports the two missing halves of the append — `extend_keyframes_mask` (`mask_utils.py:74-105`), which upstream's own docstring says EVERY appending item must call, and `clear_conditioning` (`tools.py:88-117`), which trims back to the target count and restores an ALL-ONES mask rather than the conditioned one — and lifts the last-frame keyframe as the demonstration. The attention mask is NOT the gap and no field is added for it: both ported video items pass a literal `attention_mask=None` (`keyframe_cond.py:68-76`, `reference_video_cond.py:88-96`) and the only route to a non-None mask is `ConditioningItemAttentionStrengthWrapper`, applied solely at `iclora_utils.py:169`. The sigma schedule must keep reading the TARGET count — `math.prod(latent.shape[2:])` (`schedulers.py:32 @ fd4ded7fa`) is the unpatchified target and cannot see an append — so the engine's `Ltx2SigmaSchedule(steps, video.tokens)` call, which sits AFTER the conditioning block (`src/vllm/multimodal/ltx2_video.cpp:1719 @ bc6433d1b`), re-shifts the whole schedule the moment anything appends. Reference video and generated slots stay refused: at `bc6433d1b` the reference refusal's LoRA-metadata cause is still true because PR [#938](https://github.com/mudler/vllm.cpp/pull/938) is open and unmerged | feature | | [#949](https://github.com/mudler/vllm.cpp/issues/949) | — | Nothing in the tree refuses a borrowed `vt::Tensor` that outlives the object owning its storage, and the ONLY instrument that catches one is `sanitize-cpu`, which is `continue-on-error` — that is how [#904](https://github.com/mudler/vllm.cpp/issues/904) landed. Measured in the #936 review rather than argued: with the #904 fix reverted, a plain Release build with no sanitizer runs the case 18/18 passed, 546 assertions, `rc=0`, because `dtype` lives in the `vt::Tensor` struct and not in the freed buffer, so no ordinary gate can see the dangling read. Three remedies are open and none is foregone: promote the lane once it has a `main` baseline, add a test that fails without a sanitizer, or reject the pattern statically — a prototype detector for a member access chained onto a call returning an owning type by value swept 1777 files with no hit but the defect. Anchors: the owning deleter `src/vllm/model_executor/models/ltx2_device.cpp:1088 @ 800dd082f`, the read `src/vt/cpu/cpu_layernorm.cpp:33 @ 800dd082f`. Listed under `## Owed` in [`ltx2-device-staged-view-uaf.md`](specs/ltx2-device-staged-view-uaf.md) | bug | | [#933](https://github.com/mudler/vllm.cpp/issues/933) | `ENG-EXPERT-STREAM` | Measure gateability of the `llama-cpp-unsloth` oracle by BUILDING it and RUNNING `Qwen3.8-2.4T-A95B UD-Q1_0` on it. The oracle is pinned at `36fe8e1cc` (branch `iq1-narrow`) and records `gateable = no`, because the IQ1_XXXS port is grounded in the fork's SOURCE, read and cited, which is weaker than a running comparison. It is the only place ggml type 66 is defined: the vllm.cpp pin `237ad9b96` ends at `Q1_0 = 41` and `ggml-org` master `ad1de39e0` at `Q2_0 = 42`, while type 66 carries 96.92 % of that checkpoint's parameters. Running it needs the full 370 GiB checkpoint and, per Unsloth's documentation, at least 450 GB of RAM. Until then the ported arm has no running oracle, which is what `gateable = no` makes visible | task | +| [#953](https://github.com/mudler/vllm.cpp/issues/953) | `MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation` | `POST /v1/audio/speech` silently DROPPED five keys that SGLang-Omni, serving this same model on this same route, refuses BY NAME: `temperature`, `top_p`, `top_k`, `repetition_penalty` (`request_builders.py:14-19,109-114` — this model's AR stage has ONE sampler, a fixed top-50 draw, `encoders.py:48,94-103`, so the knobs can be neither honoured nor honestly ignored) and `max_new_tokens` (`request_builders.py:56-68` — upstream's LENGTH spelling, counted in 25 Hz FRAMES rather than seconds, so a 250-frame request silently became the family's 60 s default). The identical class as [#925](https://github.com/mudler/vllm.cpp/issues/925), which cost four multi-hour runs. FIXED IN FLOW while sweeping [#672](https://github.com/mudler/vllm.cpp/issues/672) for upstream parity: all five refused by name, RED first in `test_speech_api.cpp`, two mutations both firing | bug | | [#957](https://github.com/mudler/vllm.cpp/issues/957) | `FIX-OFFLOAD-DOCS-957` | `4a183b731` (#887) turned a configured weight offload from ACCEPTED-AND-INERT into a hard startup refusal, and neither public document followed. `docs/WEIGHT-OFFLOAD.md` still said "a budget you set is accepted, reported, and does not free memory" and `docs/USAGE.md:1473` still said "Accepted and inert today", while `RefuseUnsupportedWeightOffload` (`src/vllm/model_executor/weight_offloader.cpp:72-83` @ 2daa3287f) throws from the load path (`src/vllm/entrypoints/model_loader.cpp:1410-1414` @ 2daa3287f) before any weight I/O. `ModelFactory::supports_weight_offload` defaults false and NO model sets it, so every architecture is refused; `tests/vllm/model_executor/test_weight_offloader.cpp:376-379` @ 2daa3287f asserts that count itself. Found auditing the 28 commits `documentation-checkpoint` flags: 27 needed nothing | bug | +| [#965](https://github.com/mudler/vllm.cpp/issues/965) | `MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation` | `windows-msvc-cpu`/`windows-msvc-vulkan` fail on EVERY open pull request with `C4456: declaration of 'loaded' hides previous local declaration` at `server_main.cpp:1315` — the speech engine's `loaded` nested inside the text engine's `loaded` at `:1025`, both already on `main`, and the ONLY warning in the job. **It is NOT [#645](https://github.com/mudler/vllm.cpp/issues/645)**, which is the `M_PI` regression in three LTX2 sources: a second cause hiding behind a known-red name, which is why "known-red" needs a MATCHED-ARM check and not a label. Confirmed pre-existing against three unrelated PRs (#956, #950, #939) that all fail identically. Invisible on `main` because `windows-msvc-*` are PR-only ([#584](https://github.com/mudler/vllm.cpp/issues/584)), so it presents to each author in turn as a red their own diff caused. FIXED IN FLOW while landing [#672](https://github.com/mudler/vllm.cpp/issues/672): the inner declaration is renamed, no detector weakened and no warning suppressed | bug | diff --git a/.agents/model-matrix.md b/.agents/model-matrix.md index de2cbdb13..d39115611 100644 --- a/.agents/model-matrix.md +++ b/.agents/model-matrix.md @@ -121,7 +121,7 @@ Engaged architectures (the 53 non-`INVENTORIED` rows): | 📋 | `KimiK3ForConditionalGeneration` | Kimi K3 (2.8T MoE + MoonViT-V2, DERIVE-AND-SHIP) | **W2/W5 CPU scaffolding landed** (registry stub + nested text/vision/quant config descent + text-backbone structural name-map + REFUSE-by-name forward + MXFP4-refuse loader; clean CPU build, scaffold gate 6/6). text backbone IS `KimiLinearForCausalLM` (KDA+MLA+MoE hybrid, HEAVY reuse); **does NOT fit GB10 (~1.56 TB MXFP4, ~12×)** and NOT in the pinned oracle ⇒ no on-box golden — DERIVED, proxy-gated on Kimi-Linear-48B; forward + MXFP4 + KDA delta + MoonViT-V2 not implemented (NOT-YET-BUILDABLE) | `MODEL-MM-kimi-k3-kimi-k3-for-conditional-generation` | | 📋 | `Dots3NoteForCausalLM` | dots3-note (280B-A16B multimodal MoE: DeepSeek-V3.2 text tower + sliding-window MLA, MoE ViT, `dots` audio encoder) | W0 scope only, no code. Upstream subclasses DeepSeek (`Dots3NoteModel(DeepseekV32Model)`, `Dots3NoteMoE(DeepseekV2MoE)`), so our gated MLA + DSA indexer + `noaux_tc` router + Qwen3-VL vision + Voxtral audio carry most of it; net-new is windowed MLA over 33 of 46 layers with a second latent geometry (576 vs 1088 rows), the padded/heterogeneous MLA KV spec, the headwise attention gate, the MoE ViT and the `dots` audio stem. **BEYOND-PIN** (vLLM `main` only, `#51255`, still being patched) and **oracle-memory-infeasible**: ~576 GB bf16 / ~290 GB fp8 against a 119-122 GiB ceiling on every host we own, so no performance axis is claimable and no token gate can bind until §6.4 of the spec is answered | `MODEL-MM-dots3-note-dots3-note-for-causal-lm` | | 🚧 | `MiniMaxH3DiTModel` | MiniMax-H3 (33.1B omni-modal video+audio DiT, DERIVE-AND-SHIP) | **W1/W2 landed**: packed layout (fl2va + ref2va, fp64 position grid BIT-EXACT), latent packing, euler-ancestral eta0 scheduler, and the full DiT forward all parity-gated against the UPSTREAM vLLM-Omni modules executed at reduced dimensions (**max abs diff 1.6e-7**, 10/10 cases / 2539 assertions). NOT autoregressive (no KV cache, no sampler, no logits) and **e2e HW-BLOCKED** (~354 GB checkpoint, ~133 GB/rank on 4x B300 vs 119 GiB unified); bf16 production stream + request planning + the ComfyUI-GGUF arm also landed (535 REAL tensors resolve onto our contract, geometry from shapes alone). **HW verdict CORRECTED: quantized arms FIT (~41 GB in 119 GiB)**, so e2e + speed are reachable; encoder/VAEs/audio VAE DONE (4.2e-9 vs the checkpoint's remote code); NVFP4 layout GATED as identical to ours (speed path is loader wiring); BOTH VAE DECODERS done (audio 4.2e-9, video ViT3D 8.9e-8); video tiling + 3D-CNN encoder (conditioning only) pending; encoder TEXT tower done (1.2e-7); **serving `/v1/videos` DONE and the DEVICE-RESIDENT forward (W2b, f32) LANDED + GPU-VERIFIED on Thor sm_110 at video 1.49e-7 / audio 8.94e-8**; bf16 stream + fusion folds + the FP4 path (needs sm_121a) + a real-checkpoint run pending. **2026-08-05: the AUDIO-VAE ENCODER is ported** (DAC analysis stack + `pre_block` AttnProjection + `mean_proj`, gated stage by stage vs the checkpoint's own remote code at 2.98e-8 / 1.64e-7 / 1.86e-8) with its own checkpoint loader gated on the real 1087-tensor manifest — so **ref2va AUDIO and VIDEO+AUDIO references are now WIRED** (audio rows move by 0.51 / 0.71; a different waveform still moves them by 7.1e-4). Both VAEs are now complete in both directions. **bf16 13-SHARD RELEASE INDEXES 2026-08-07 (`row/H3-BF16-SHARDED-DIT`)**: `MiniMaxH3ShardedCheckpoint` resolves the ORIGINAL 66.3 GB release through its own `model.safetensors.index.json` (a tensor named in the index but missing from its shard throws BY NAME), `EnumerateMiniMaxH3ShardedTensors` feeds the shared shapes-only geometry parser, and `LoadMiniMaxH3DitFromShards` is the host-f32 reference loader. Gated CPU-only at 72/72/54497 (post-rebase): every tensor resolves to the shard the index named AND to the bytes written there, the derived geometry equals the single-file path field for field, and a SPARSE 13-shard release with the REAL 535 tensors at REAL shapes (66.3 GB declared, 144 KB on disk) derives the SHIPPED geometry (50/5376/56/128/14336/24/32/1x2x2/5120). **STREAMS 2026-08-07 (`row/H3-BF16-SHARDED-STREAM`)**: `StreamMiniMaxH3ShardedToDeviceBf16` uploads it one tensor at a time — a BF16 tensor bound for a bf16 device slot goes straight from the mmap with ZERO host buffer, so peak host is bounded by ONE tensor (observed `host_peak=8192`, `direct=37 converted=9`); bit-exact vs the non-streamed `StageMiniMaxH3DitWeights` reference over all 46 views with identical logits, rope.inv_freq host-resident, 73/73/55203. Spec §8.14. **bf16 TEXT ENCODER + THE CONDITIONING NUMBER 2026-08-07 (`row/H3-ENC-BF16-COND-DIFF`)**: the 14-shard 63 GB bf16 Qwen3-VL-32B encoder streams to device too (`StreamMiniMaxH3EncoderShardsToDevice`, q/k/v and gate/up fused ON DEVICE), `--encoder-only` runs the tower alone (peak ~96 -> ~49 GiB by not loading the DiT first), and the widening is gated BIT-IDENTICAL vs an f32-staged tower so the A/B cannot be confounded. MEASURED on Thor over 233 tokens: Q4_K_M vs bf16 conditioning is cos 0.99745 mean / 0.909 min, rel RMS 6.85% excluding the attention sink, median rotation 3.5 deg — same energy as a ONE-WORD prompt edit but DIFFUSE (232/233 tokens rotate vs 172/233). Whether the RENDER changes is NOT established. 75/75/55609. Spec §8.15. This UNBLOCKS the quantization-quality question; no bf16-vs-quant render or speed number is claimed. Spec §8.13. **W-FP4a LANDED (CPU) 2026-08-06 (`row/H3-FP4-SPEED`)**: the device DiT forward now routes the NVFP4 projections through the shared Marlin W4A16 dispatcher (fp4 kept packed; no new quant code), fp4-vs-bf16 wiring gate GREEN (62/62·30039). **W-FP4a GB10 leg LANDED 2026-08-06 (`row/H3-FP4-GPU-E2E`, PR #64):** on sm_121a the Marlin W4A16 path RAN for all 11 projections (`dense_gemms==11` default — VT_MARLIN_DENSE is default-ON → vLLM's own DENSE Marlin GEMM, not the grouped route; `marlin_gemms==11` under VT_MARLIN_DENSE=0; `fallback_gemms==0`), fp4-vs-bf16 BYTE-EXACT (max\|diff\|=0), and the fp4 arm is a MEMORY win not a diffusion-forward speed win (per-forward bf16/fp4 3.47× @seq64 → 0.79–0.83× @seq4224–7040; ~16 vs ~66 GB device). Real-checkpoint fp4-resident t2va e2e RUNS (real 18.75 GB NVFP4 DiT + VAEs + GGUF Qwen3-VL-32B encoder → valid mp4/wav; DiT s/step 5.45/20.0/209 s @512/768/REF-209f) but frames are a non-scene patch-grid at 12/20/50 steps → OPEN render bug (device VAE/denoise). vLLM-Omni has no quantized H3 arm (BF16-only) so any comparison is HW/loader-forced-indirect — spec §8. **ONE-SURFACE ROW 2 LANDED 2026-08-08 (`row/H3-VIDEO-ABI`)**: the whole assembly pipeline is library-owned (`vllm::multimodal::MiniMaxH3VideoEngine`, `minimax_h3_video.cpp`) behind the ABI v12 `vllm_video_*` entry points; `/v1/videos` routes through the SAME seam; `minimax_h3_gen`+`minimax_h3_mux` are thin `vllm.h` clients, frames+WAV byte-identical to the pre-fold binary on the committed fold fixture (`test_minimax_h3_video_fold` 3-arm gate + the v12 `test_capi` section); GB10 real-video re-verify via the v12 ABI = named residual. **ROW 2 DEVICE-SEAM FOLLOW-UP (#135; replaces #134):** ABI 0/1 maps once to `vt::DeviceType`; shared code dispatches through `GetBackend(device_type)`, restoring DSR 34→32 without a baseline/allowlist change; CPU compile/fold test pending in CI due shared-disk pressure **PRUNED CHECKPOINTS LOAD AND RUN 2026-08-10 (`row/H3-PRUNED`, #241, spec §8.21):** the community `pruned` variants (`unsloth/MiniMax-H3-GGUF` Q2_K..Q8_0, `lilcheaty/MiniMax-H3-NVFP4` `*_pruned_nvfp4`) are not lossily pruned but ComfyUI's AdaLN timestep-CURVE refactor (`comfy/ldm/minimax/model.py:419-432,610-615`): an `adaln_t_table` [1025, 8] lerp replaces the sinusoidal+MLP time embedder, no SiLU before the AdaLN linear, and its in_features drop 2688 -> 8, collapsing `adaln_proj` from 13.04B of 33.12B to 0.04B and the DiT to 20.11B — so a pruned Q8_0 (21.4 GB) costs what our unpruned Q4_K_M (19.9 GB) costs. Gated on the REAL 532-tensor pruned GGUF manifest (532 = 535 - 4 `time_embedder.*` + 1 table, header-only), the clamped-lerp golden, and a CONSTRUCTED-curve test proving the pruned forward is numerically identical to the unpruned one; 79/79/57299. | `MODEL-DIFFUSION-minimax-h3-mini-max-h3-dit` | -| 🚧 | `MiniMaxMusic3ForConditionalGeneration` | MiniMax-Music3 (text-to-music: ~8.6B Qwen3 global LLM + 0.646B 8-codebook RVQ depth decoder + 2.4B fp32 flow-matching DiT + 0.054B DAC Flow-VAE) | **W0 DONE + W1 LANDED (#672, [spec](specs/minimax-music3.md)).** Diffusers oracle gateable with committed per-stage goldens (#708); modular loader in the tree, 1413/1413 assertions against the real checkpoint (#714). Geometry MEASURED from every component `config.json` plus safetensors headers by range request. First row to exercise the fallback-oracle rule (#647): absent from vLLM AND vLLM-Omni, so its primary oracle is the OPEN diffusers PR #14456 head `c6da9936` with SGLang-Omni `748a0b43` as the e2e cross-check. **W2/W3 LANDED**: the AR half computes and is gated. **W4/W5 LANDED**: the ACOUSTIC half — flow-matching DiT, `FlowMatchEulerDiscreteScheduler` with `invert_sigmas`, the CFG mix, the denoise loop's window bookkeeping and the DAC Flow-VAE vocoder over the shared `vocoder1d` primitives — gated at reduced dimensions (27 cases / 265 assertions, no checkpoint) AND at full scale against the committed capture (scheduler BIT-EXACT 22 016/22 016; waveform 88 064 samples, 0 outside a tolerance calibrated against torch reproducing its own goldens). **W6 LANDED**: registered as a `SpeechRegistry` family (`minimax-music3`, detection INSPECTS `modular_model_index.json`), `SpeechGenParams` extended ADDITIVELY with `lyrics`/`description` plus three generation controls (IndexTTS-2.5 byte-identical, 0 lines changed), the new `vllm_speech_*` C-ABI surface at v20, and `POST /v1/audio/speech` on the example server as a thin ABI client. The denoise + decode composition reproduces the capture's `waveform.npy` — 88 064 values, 0 outside tolerance, max abs-diff 4.52e-06 — and the WAV payload is BIT-EXACT against the quantization of that golden. **W7 LANDED: ONE ARM IMPLEMENTED AND VALUE-GATED, the rest refused.** The RVQ depth decoder loads from GGUF Q4_K (`audio-cpp/MiniMax-Music3-GGUF`@`c36aaeed`, `rvq_depth_decoder_q4_k.gguf`, 405 752 480 B, sha256 `4c5d41b2...c70cbdd0` — pinned because an unpinned quantized checkpoint is not reproducible): 47 tensors as 36 Q4_K + 9 BF16 + 2 F16, dequantized to bf16 through the SHARED `gguf_dequant.h` seam, lineage keyed on `audiocpp.model_spec.family` because `general.architecture` reads `audiocpp`/`mm3`/`qwen3`/`wan` for this ONE model and `wan` collides with real Wan video GGUFs. Gated 6 cases / 319 assertions against the W3 golden's 716 800 values at a bound DERIVED rather than chosen: worst per-tensor relative L2 0.0742 (bound 0.10), full-scale mean abs-diff 0.0324 / max abs-diff 0.3125 / 2.84% bit-identical. **THE GATE-DESIGN FINDING, which generalizes past this row: an upper-bound-only tolerance CANNOT distinguish a real quantized arm from a silent dequant fallback, because the fallback is CLOSER to the golden** — bf16 weights through the identical forward give mean abs-diff 0.00182, 17.8x nearer than the genuine Q4_K arm, so every plausible upper bound PASSES the failure. What catches it is a LOWER bound (5e-3, sitting between the two), gated by a positive control that actually runs the fallback and asserts the gate rejects it. Two further INDEPENDENT legs prove the quantized path was taken rather than inferring it: the RESIDENT ggml type of every tensor (loader bookkeeping — localizes a fault to a name, but a loader that lied about its tallies would pass it, which is why mutation QM2 exists) and the Q4_K LATTICE (a property of the DATA needing no loader cooperation — 0 of 524 288 32-element windows exceed 16 distinct values, against a bf16 control from the same file at 127 of 128). Second finding: the 2 F16 islands do NOT round-trip exactly (rel 3.23e-08 / 2.36e-08) because the quantizer re-encoded them BF16->F16 and F16's exponent range is NARROWER, flushing weights below ~6e-08 to zero — so the gate splits the islands, BF16 bit-exact and F16 bounded; asserting "unquantized means exact" for all 11 REDS a correct reader. 18 mutations across both layers all fire, one of them (QM4, the lineage guard) only after a COVERAGE HOLE it exposed was closed with synthetic GGUF fixtures. The 4 OTHER components' GGUF arms, the `mm3` and ComfyUI lineages, and every non-GGUF format are REFUSED BY NAME and owed. Quantized MiniMax-Music3 checkpoints DO exist — a HuggingFace survey on 2026-08-14 found 14 community repositories in 5 formats (GGUF of all five components, int8/w4a8, MLX, plus a proprietary Cortiq arm), recorded query-by-query with result counts in spec §9.1. `minimax_music3_quant.{h,cpp}` is a SEPARATE translation unit per porting-a-model.md that diagnoses 8 formats at the THREE places a quantized checkpoint announces itself (tree `.gguf`, manifest sidecars plus the dtype-only fp8/int8 cases, and `quantization_config`/MLX `quantization`) and refuses each BY NAME with the evidence, the missing piece, the supported arm, the phase and the issue. 29 cases / 125 assertions without a checkpoint plus 6 cases / 319 assertions against the pinned artifact, RED first at 8/8 probe checks, 18 of 18 mutations fire; every tolerance is DERIVED from a printed measurement rather than chosen, and no speed number is claimed. GGUF headers MEASURED by range request (56 MiB, no weight bytes): "the GGUF arm" is THREE MUTUALLY INCOMPATIBLE LINEAGES and `general.architecture` cannot separate them (`audiocpp`/`mm3`/`qwen3`/`wan`, and `wan` collides with real Wan video GGUFs), the ComfyUI lineage ships the DiT + condition encoder ONLY so it can never generate audio, and `comfy.gguf.orig_shape.*` is absent from all ten files. The 8.6B **W2's remainder LANDED (this PR): the 8.6B `Qwen3ForCausalLM` forward runs.** `Qwen3DenseModel::ForwardEmbeds` gives the landed dense path the `inputs_embeds` entry its multimodal siblings already had (`qwen3_vl.h:145,159`, `gemma4.h:210-218`, `muse_glimmer.h:369-380`), additively and proved BIT-IDENTICAL to the token-id `Forward` in logits AND in the paged KV it writes. Teacher-forced on the capture's own codes, 25 steps reproduce `frame_hiddens[:, :4096]` - 102 400 values, 9337 bit-identical, mean abs-diff 1.763e-02 - INSIDE a matched control (upstream's own model under `sdpa_kernel(MATH)`: 12 036 bit-identical, mean abs-diff 1.475e-02), with the per-step error FLAT rather than compounding. The oracle's OWN sampled semantic codes rank 2.48 on average under our guided logits where chance is 8191.5, and the gate carries its own NEGATIVE CONTROL: a one-step-shifted alignment reads mean abs-diff 0.803, 46x outside. **The e2e case at `POST /v1/audio/speech` is OBSERVED TO PASS (2026-08-15, #852)**: 200 `audio/wav`, 12332 bytes, 2 AR frames -> 6 latent frames -> 3072 samples/channel of 44100 Hz stereo, 6144 int16 samples all non-zero, 0 clipped, 2818 of 3072 positions differing between L and R; 21 assertions in 7:54, and 5 cases / 535 assertions for the whole file in 31:14. It had never been seen because the request body posted `audio_duration_s`, the FIELD name, where `ParseSpeechRequest` reads `audio_duration` - so the key was silently dropped and the family's 60 s default substituted, making every run a ~750x job (1500 AR frames not 2, 8 denoise windows not 1, 5167 vocoder latents not 6) that could never have passed its own payload assertion. The near-miss key is now REFUSED (#925). **The earlier diagnosis was wrong and the way it was wrong generalizes**: open LM shard fds do NOT locate the program counter, because `LoadBf16Direct` BORROWS the mapping and a borrowed `OwnedTensor` keeps it alive for the whole request, while the depth decoder's tensors are COPIED so its fd closes on return. A symbol-resolved profile puts the LM weight load at 180 s (1 thread, state `D`, ~92 MB/s off the NAS) in BOTH binaries, and the cost in `LinearNoBias` 42-57% / `Threadpool::Barrier` 25% / `vocoder1d::ConvTranspose1d` 88.5%; the suspected 200000 x 4096 `lm_head` transpose is 1.388 s. The GGUF k-quant ARM for the remaining components is still owed; spec §5's token-exact gate is WITHDRAWN because upstream's AR stage has no greedy path | `MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation` | +| 🚧 | `MiniMaxMusic3ForConditionalGeneration` | MiniMax-Music3 (text-to-music: ~8.6B Qwen3 global LLM + 0.646B 8-codebook RVQ depth decoder + 2.4B fp32 flow-matching DiT + 0.054B DAC Flow-VAE) | **W0 DONE + W1 LANDED (#672, [spec](specs/minimax-music3.md)).** Diffusers oracle gateable with committed per-stage goldens (#708); modular loader in the tree, 1413/1413 assertions against the real checkpoint (#714). Geometry MEASURED from every component `config.json` plus safetensors headers by range request. First row to exercise the fallback-oracle rule (#647): absent from vLLM AND vLLM-Omni, so its primary oracle is the OPEN diffusers PR #14456 head `c6da9936` with SGLang-Omni `748a0b43` as the e2e cross-check. **W2/W3 LANDED**: the AR half computes and is gated. **W4/W5 LANDED**: the ACOUSTIC half — flow-matching DiT, `FlowMatchEulerDiscreteScheduler` with `invert_sigmas`, the CFG mix, the denoise loop's window bookkeeping and the DAC Flow-VAE vocoder over the shared `vocoder1d` primitives — gated at reduced dimensions (27 cases / 265 assertions, no checkpoint) AND at full scale against the committed capture (scheduler BIT-EXACT 22 016/22 016; waveform 88 064 samples, 0 outside a tolerance calibrated against torch reproducing its own goldens). **W6 LANDED**: registered as a `SpeechRegistry` family (`minimax-music3`, detection INSPECTS `modular_model_index.json`), `SpeechGenParams` extended ADDITIVELY with `lyrics`/`description` plus three generation controls (IndexTTS-2.5 byte-identical, 0 lines changed), the new `vllm_speech_*` C-ABI surface at v20, and `POST /v1/audio/speech` on the example server as a thin ABI client. The denoise + decode composition reproduces the capture's `waveform.npy` — 88 064 values, 0 outside tolerance, max abs-diff 4.52e-06 — and the WAV payload is BIT-EXACT against the quantization of that golden. **W7 LANDED: ONE ARM IMPLEMENTED AND VALUE-GATED, the rest refused.** The RVQ depth decoder loads from GGUF Q4_K (`audio-cpp/MiniMax-Music3-GGUF`@`c36aaeed`, `rvq_depth_decoder_q4_k.gguf`, 405 752 480 B, sha256 `4c5d41b2...c70cbdd0` — pinned because an unpinned quantized checkpoint is not reproducible): 47 tensors as 36 Q4_K + 9 BF16 + 2 F16, dequantized to bf16 through the SHARED `gguf_dequant.h` seam, lineage keyed on `audiocpp.model_spec.family` because `general.architecture` reads `audiocpp`/`mm3`/`qwen3`/`wan` for this ONE model and `wan` collides with real Wan video GGUFs. Gated 6 cases / 319 assertions against the W3 golden's 716 800 values at a bound DERIVED rather than chosen: worst per-tensor relative L2 0.0742 (bound 0.10), full-scale mean abs-diff 0.0324 / max abs-diff 0.3125 / 2.84% bit-identical. **THE GATE-DESIGN FINDING, which generalizes past this row: an upper-bound-only tolerance CANNOT distinguish a real quantized arm from a silent dequant fallback, because the fallback is CLOSER to the golden** — bf16 weights through the identical forward give mean abs-diff 0.00182, 17.8x nearer than the genuine Q4_K arm, so every plausible upper bound PASSES the failure. What catches it is a LOWER bound (5e-3, sitting between the two), gated by a positive control that actually runs the fallback and asserts the gate rejects it. Two further INDEPENDENT legs prove the quantized path was taken rather than inferring it: the RESIDENT ggml type of every tensor (loader bookkeeping — localizes a fault to a name, but a loader that lied about its tallies would pass it, which is why mutation QM2 exists) and the Q4_K LATTICE (a property of the DATA needing no loader cooperation — 0 of 524 288 32-element windows exceed 16 distinct values, against a bf16 control from the same file at 127 of 128). Second finding: the 2 F16 islands do NOT round-trip exactly (rel 3.23e-08 / 2.36e-08) because the quantizer re-encoded them BF16->F16 and F16's exponent range is NARROWER, flushing weights below ~6e-08 to zero — so the gate splits the islands, BF16 bit-exact and F16 bounded; asserting "unquantized means exact" for all 11 REDS a correct reader. 18 mutations across both layers all fire, one of them (QM4, the lineage guard) only after a COVERAGE HOLE it exposed was closed with synthetic GGUF fixtures. The 4 OTHER components' GGUF arms, the `mm3` and ComfyUI lineages, and every non-GGUF format are REFUSED BY NAME and owed. Quantized MiniMax-Music3 checkpoints DO exist — a HuggingFace survey on 2026-08-14 found 14 community repositories in 5 formats (GGUF of all five components, int8/w4a8, MLX, plus a proprietary Cortiq arm), recorded query-by-query with result counts in spec §9.1. `minimax_music3_quant.{h,cpp}` is a SEPARATE translation unit per porting-a-model.md that diagnoses 8 formats at the THREE places a quantized checkpoint announces itself (tree `.gguf`, manifest sidecars plus the dtype-only fp8/int8 cases, and `quantization_config`/MLX `quantization`) and refuses each BY NAME with the evidence, the missing piece, the supported arm, the phase and the issue. 29 cases / 125 assertions without a checkpoint plus 6 cases / 319 assertions against the pinned artifact, RED first at 8/8 probe checks, 18 of 18 mutations fire; every tolerance is DERIVED from a printed measurement rather than chosen, and no speed number is claimed. GGUF headers MEASURED by range request (56 MiB, no weight bytes): "the GGUF arm" is THREE MUTUALLY INCOMPATIBLE LINEAGES and `general.architecture` cannot separate them (`audiocpp`/`mm3`/`qwen3`/`wan`, and `wan` collides with real Wan video GGUFs), the ComfyUI lineage ships the DiT + condition encoder ONLY so it can never generate audio, and `comfy.gguf.orig_shape.*` is absent from all ten files. The 8.6B **W2's remainder LANDED (this PR): the 8.6B `Qwen3ForCausalLM` forward runs.** `Qwen3DenseModel::ForwardEmbeds` gives the landed dense path the `inputs_embeds` entry its multimodal siblings already had (`qwen3_vl.h:145,159`, `gemma4.h:210-218`, `muse_glimmer.h:369-380`), additively and proved BIT-IDENTICAL to the token-id `Forward` in logits AND in the paged KV it writes. Teacher-forced on the capture's own codes, 25 steps reproduce `frame_hiddens[:, :4096]` - 102 400 values, 9337 bit-identical, mean abs-diff 1.763e-02 - INSIDE a matched control (upstream's own model under `sdpa_kernel(MATH)`: 12 036 bit-identical, mean abs-diff 1.475e-02), with the per-step error FLAT rather than compounding. The oracle's OWN sampled semantic codes rank 2.48 on average under our guided logits where chance is 8191.5, and the gate carries its own NEGATIVE CONTROL: a one-step-shifted alignment reads mean abs-diff 0.803, 46x outside. **The e2e case at `POST /v1/audio/speech` is OBSERVED TO PASS (2026-08-15, #852)**: 200 `audio/wav`, 12332 bytes, 2 AR frames -> 6 latent frames -> 3072 samples/channel of 44100 Hz stereo, 6144 int16 samples all non-zero, 0 clipped, 2818 of 3072 positions differing between L and R; 21 assertions in 7:54, and 5 cases / 535 assertions for the whole file in 31:14. It had never been seen because the request body posted `audio_duration_s`, the FIELD name, where `ParseSpeechRequest` reads `audio_duration` - so the key was silently dropped and the family's 60 s default substituted, making every run a ~750x job (1500 AR frames not 2, 8 denoise windows not 1, 5167 vocoder latents not 6) that could never have passed its own payload assertion. The near-miss key is now REFUSED (#925). **The earlier diagnosis was wrong and the way it was wrong generalizes**: open LM shard fds do NOT locate the program counter, because `LoadBf16Direct` BORROWS the mapping and a borrowed `OwnedTensor` keeps it alive for the whole request, while the depth decoder's tensors are COPIED so its fd closes on return. A symbol-resolved profile puts the LM weight load at 180 s (1 thread, state `D`, ~92 MB/s off the NAS) in BOTH binaries, and the cost in `LinearNoBias` 42-57% / `Threadpool::Barrier` 25% / `vocoder1d::ConvTranspose1d` 88.5%; the suspected 200000 x 4096 `lm_head` transpose is 1.388 s. The GGUF k-quant ARM for the remaining components is still owed; spec §5's token-exact gate is WITHDRAWN because upstream's AR stage has no greedy path **PARITY SWEEP + MUSIC-ONLY SERVER LANDED (#672, #953)**: upstream's whole user-facing surface was enumerated field by field (SGLang-Omni `748a0b43` `models/minimax_music3/`, diffusers `c6da9936` `modular_pipelines/minimax_music3/`) and recorded with `file:line` in spec §10.1. **`--model` is now OPTIONAL when `--speech-model` is given**, so `vllm-server --speech-model ` serves `/v1/audio/speech` and registers NOTHING else — upstream's own shape (`sgl-omni serve --model MiniMaxAI/MiniMax-Music3`, `config.py:29-63`: three stages, no chat LLM), and the third instance of the task-conditional dispatch already in `server_main.cpp` for pooling and transcription-only checkpoints (vLLM `api_server.py:255-265`). ADDITIVE and proved: the ONLY case whose verdict changes is BOTH flags absent, which was an error and remains one with a message naming both options; the route table is gated in BOTH directions over a REAL SOCKET. New `examples/minimax_music3_gen` — the music family was the one generative family in this tree with NO example — a THIN C-ABI client (`vllm_speech_engine_load` + `vllm_synthesize`, `include/vllm.h` only). **Five keys upstream refuses BY NAME were SILENT here** and are now refused (#953): `temperature`/`top_p`/`top_k`/`repetition_penalty` (`request_builders.py:14-19,109-114`; the AR stage has ONE sampler, a fixed top-50 draw) and `max_new_tokens` (`request_builders.py:56-68`; upstream's length spelling in 25 Hz FRAMES, not seconds) — the #925 class exactly. **The e2e gate no longer reports a SKIP WEARING A PASS**: it read 5 cases / 5 passed with `assertions: 0` when the checkpoint was absent, the same shape that fooled this project on `test_qwen3_paged_engine`. Split into a CHECKPOINT-FREE half that runs unconditionally in CI (request contract, both ceilings, the speech-only route table over a real socket with a stub synthesizer) and the env-gated checkpoint half, whose HTTP case now runs over a REAL SOCKET against the music-only server shape rather than calling `handle_audio_speech`; a coverage-report case prints which arms ran every run. No-env arm 9 cases / 37 assertions where it was 5 / 0. **The WEIGHTS are documented** per `porting-a-model.md` §2.1: `docs/USAGE.md` carries component-by-component tables for the diffusers arm at `MiniMaxAI/MiniMax-Music3`@`fbdf52fbaaca799592917417eb05f1899f1255ec` (**28.5 GB resident, 28 517 617 303 B MEASURED**, out of a 57.4 GB repository, and why they differ), the native `.pth` arm we refuse and SGLang-Omni serves, the one implemented GGUF Q4_K artifact with its sha256, and the 14 THIRD-PARTY quantized repositories in 5 formats each marked refused. The revision is VERIFIED not copied: `condition_encoder/diffusion_pytorch_model.safetensors` on disk hashes to `83179c5e…a202c2a4d`, that revision's own LFS record. OWED and named rather than discovered: the non-`wav` response formats (no encoder vendored; note upstream DOWNMIXES TO MONO for every non-wav format, `client/audio.py:328-334`), request batching and `/v1/audio/speech/batch` (SGLang runs continuous batching at 16 with TWO engine rows per request for the CFG twin, `engine_builder.py:74-77`), and the 32 kHz delivery resample. Streaming is a PERMANENT refusal, not a debt: neither upstream arm has it (`supports_streaming_vocoder=False`). One place we are AHEAD of both arms: `guidance_scale` is a real per-request control here, where diffusers freezes it at 1.7 into the guider component (`denoise.py:180`) and SGLang exposes it only as a serve-time knob | `MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation` | | 📋 | `LTX2VideoTransformer3DModel` | LTX-2.5 (21.00B joint video+audio flow-matching DiT, Lightricks) | **L0 spec committed 2026-08-11 (#435, [spec](specs/ltx-2-5.md)).** Geometry MEASURED from the FP8 checkpoint's own safetensors header by HTTP range request (6124 tensors, 881,048-byte header, no payload downloaded): **21.00B** params — 48 blocks @ 386.7M = 18.560B, audio embeddings connector 2.016B, global 0.427B. The filename says `22b` and the Diffusers card says ~19B; the MEASURED count is what this row uses. Video stream 4096 (32 heads x 128), audio stream 2048 (32 heads x 64), in/out channels 128 both. Per block SIX attentions — `attn1` (video self), `attn2` (video<->text, cross 4096), `audio_attn1`, `audio_attn2` (cross 2048), and the two CROSS-MODAL `audio_to_video_attn` / `video_to_audio_attn` — which is the structural break from MiniMax-H3: H3 packs every modality into ONE sequence with per-row token tags, LTX runs TWO streams coupled by explicit cross-attention. **Per-head gated attention on every attention** (`to_gate_logits` = `Linear(query_dim, heads, bias=True)`, `attention.py:513-514`, applied AFTER the attention output at `:577`) — H3 has no analogue and getting it wrong renders plausibly-wrong rather than erroring. FFN is gelu-approximate 4096->16384->4096 with **NO bias** while `audio_ff` 2048->8192->2048 **HAS** bias, which independently confirms `ff_bias=false` / `audio_ff_bias=true` from `model_configurator.py:78-80` — checkpoint and source agreeing, not either alone. **RETRACTED 2026-08-12, was billed as a FREE WIN.** The spec claimed 2.5 sets `use_prompt_adaln_single=false` so the cross-attention K/V are timestep-free and cacheable. The shipped checkpoint DISPROVES it: it carries 12 `prompt_adaln_single`/`audio_prompt_adaln_single` tensors including a `timestep_embedder.linear_1 [4096, 256]` (256 = the sinusoidal timestep width), and `model.py:223-227` builds that module ONLY when the flag is TRUE. `transformer.py:441` was quoted as proof of no timestep term, but `:442-443` add one whenever `prompt_timestep` is not None, and the comment above them says exactly that. NO SHIPPED DEFECT: `ltx2_dit.cpp:672` refuses the cache by name when the flag is on, so the feature is correct-and-inapplicable rather than silently wrong, and stays gated bit-identical and prompt-bound for any checkpoint that does set it false. **ORACLE:** vLLM-Omni does NOT support 2.5 — `resolve_ltx_pipeline_recipe` keys on 2 / 2.3 only and RAISES otherwise (`ltx2_recipes.py:162-166`), with upstream [vllm-omni#6066](https://github.com/vllm-project/vllm-omni/issues/6066) filed 2026-08-11 — but its `DiffusersAdapterPipeline` is generic (`DiffusionPipeline.from_pretrained`, `pipeline_diffusers_adapter.py:116`), so vLLM-Omni CAN execute 2.5 via `--load-format diffusers`. Binding oracle = that adapter; immediate cross-check = Lightricks `ltx-pipelines`. **HW: FITS ONE GB10** at ~29 GB (NVFP4 DiT 18.72 + NVFP4 Gemma-4 TE 7.40 + VAEs 1.83 + upscaler 1.00) vs H3's ~41 GB. **OWED UP FRONT, not to be discovered later:** the speed axis lands `PENDING` because the adapter is a black box (`supports_step_execution=False`, `supports_request_batch=False`) and therefore NOT vLLM's production configuration, which AGENTS.md requires as the denominator; DiffVAE (`NADiffusionDecoder`, neighborhood attention) is REFUSED BY NAME until its own row rather than silently downgraded to the Conv VAE; and no render-quality claim is made from structural e2e. | `MODEL-DIFFUSION-ltx-2-5-ltx2-video-transformer-3d-model` | | ✅ | `LagunaForCausalLM` | Poolside Laguna-S-2.1 (118B/8B MoE) | **LONG-CTX DECODE LEVERS LANDED + MEASURED (2026-08-03, `CLAIM-LAGUNA-LONGCTX-LEVERS`): window-bounded SWA reads (`VT_LAGUNA_SWA_WINDOW`, default-ON, BYTE-EXACT) bound the four `DecodeAttnGqa*` kernels' read to the ~512 sliding window (vLLM `laguna.py:412`) — GB10 A/B token-IDENTICAL `=1` vs `=0` at 520-token context (truncation active), MEASURED −0.30 ms/step at ~2k (~0 at ≤512, grows linearly). bf16 paged KV (`VT_LAGUNA_KV_BF16`, default-OFF opt-in) a distributional near-tie left UNRATIFIED. See BENCHMARKS `CLAIM-LAGUNA-LONGCTX-LEVERS`.** — **NVFP4 W4A4 ARM RAN on GB10 (N4, 2026-08-01, `CLAIM-LAGUNA-NVFP4-N4`): the additive safetensors NVFP4 arm (N1a/N1b/N2/N3 — `Nvfp4Weight` expert fields + `LoadLagunaForCausalLMWeights` + `LqGemmNvfp4Fp4` per-expert TRUE-W4A4 + `LagunaFfnBlock` `fp4` branch + `laguna_gen` dir-autodetect; CPU-gated `test_laguna_nvfp4_loader` 3/3·61, GGUF path byte-identical) generates COHERENTLY on the real 67 GiB `poolside/Laguna-S-2.1-NVFP4`. vs the vLLM MARLIN golden (vLLM's exact prompt ids injected): FIRST 2 TOKENS MATCH exactly, then near-tie divergence (our TRUE-W4A4 fp4-activations vs the MARLIN golden's W4A16 bf16-activations — different precision, EXPECTED; shares golden vocab). SPEED (N5, trace-driven, 2026-08-01): 0.16 → ~4.5 tok/s (~28× THIS SESSION), now ~4× from vLLM 18.8. **Lever #2** (nsys found the bf16 tower running host `MatmulNK` on the CUDA queue): route it to the GPU (`LqGemm` bf16 → `CastBf16` + `MatmulBT`, weight stays bf16) → 6.34 → 0.39 s/tok (16×). **Lever #1** (nsys found the emulation expert GEMM at 92%, GPU 87% busy): the engine's native sm120a fp4 tensor-core MMA (`MatmulNvfp4Fp4Native`) reads the SAME linear scales — it was gated OFF behind `VT_NVFP4_FP4_NATIVE`; default it ON in the driver → 0.39 → ~0.20-0.24 s/tok (~2×). Both coherent + near-tie (byte-identical ids to emulation; first token matches golden). Two GB10 memory fixes landed to run (shard-release + context-before-load). OPEN #234 (remaining ~4×): grouped W4A4 MoE (top_k×3 launches → 3), `ResidentNvfp4`, decode CUDA-graph + on-GPU sampling (the host-orchestration tail). Spec `.agents/specs/laguna-nvfp4-arm-2026-07-31.md` §N4/§N5. The GGUF-Q4_K track (below) is the separate keep-quant vehicle.** Prior **FASTER DECODE (W9, 2026-07-31, `CLAIM-LAGUNA-W9-GROUPED`): the 30 un-grouped per-expert keep-quant GEMV launches/step (top_k × {gate,up,down} `LqGemmRowSlice`) fold onto the SHARED `vt::MatmulBTQuantGrouped` op — per token, Pk experts' gate/up/down each collapse to ONE grouped launch over the already-stacked `[E*N,H]` tower (no loader change). Same-binary A/B on real UD-Q4_K_XL (GB10, `--gpu`, drop_caches cold, 24 tok): grouped (`VT_LAGUNA_GROUPED_MOE=1`, default) == per-expert (`=0`) BYTE-IDENTICAL (md5 `754728c6`, both == W6 golden) + decode 0.18 → 0.13 s/tok (1.38×). Routes through the shared vt op (fold policy). Cumulative with W8: decode 0.66 → 0.13 s/tok (5.1×; 1.5 → 7.7 tok/s; 18× → 3.6× vs llama.cpp 27.8). Next lever: device-resident decode (#1). See spec §W9.** Prior **FASTER DECODE (W8, 2026-07-31, `CLAIM-LAGUNA-W8-EMBED`): `LagunaEmbed` no longer converts the whole 1.23 GB embed table to f32 every token (it gathered T rows out of the whole [Vsz,H] table via `ReadF32` — ~311M host element-converts/token, the DOMINANT decode cost the W7 profile under-filed as "#5"); now gathers only the T needed rows directly (BIT-IDENTICAL — same per-element conversion, same rows). GATED on the real 3-shard UD-Q4_K_XL GGUF (GB10, `--gpu`, W6 cached, drop_caches cold, 24 tok): TOKEN-IDENTICAL to the W5/W6 golden (`22345 83 350 785 …`, coherent " Paris.") + decode 0.66 → 0.17 s/tok = 3.9× (1.5 → 5.9 tok/s; 18× → 4.7× vs llama.cpp 27.8). See `.agents/specs/laguna-s21-w7-speed-2026-07-31.md` §W8. Next: grouped-expert GEMM (=A3) then device-resident decode.** Prior **DECODE-SPEED ATTRIBUTED (W7 profile-only, 2026-07-31, `CLAIM-LAGUNA-W7-SPEED`): `nsys` of the W6 decode (real UD-Q4_K_XL GGUF, GB10) attributes the 0.66 s/tok (~1.5 tok/s vs llama.cpp 27.8 on identical bytes, ~15-18x) to HOST-ORCHESTRATION, not kernel compute — GPU active only 32.7% of the step, 67.3% host/idle; 22,115 `cudaStreamSynchronize` (~2,764/step, zero GPU overlap) from the ~1,795 per-GEMM `DrainQueue` in `LagunaForwardGgufCached` + scalar host glue; 39.4% of GPU time is `QuantizeQ8K` activation-quant (per-GEMM), weight GEMVs un-grouped at ~22% of the 240 GB/s peak (llama.cpp ~76%); no H2D/D2H (unified memory). Ranked levers (all in-tree from ds4): device-resident decode 1.5->~5-7 tok/s, grouped-expert GEMM (`MatmulBTQuantGrouped`) +1.5-2x + dedupes the activation-quant, decode CUDA-graph, tuned MMVQ; + free host cleanups (`LagunaEmbed` copies the whole 1.23 GB embed table/token, per-token RoPE-cache rebuild). Honest reachable ~13-20 tok/s, 27.8 a stretch. NO code changed. See `.agents/specs/laguna-s21-w7-speed-2026-07-31.md`. Prior RUNNABLE + FAST DECODE (W6, 2026-07-31): a per-layer K/V cache + single-token incremental decode replaces W5's O(n²) STATELESS recompute — TOKEN-IDENTICAL (byte-equal ids, md5 `754728c6…` match, == the W5 golden) and 5.05× faster per token: decode 3.33 → 0.66 s/tok on the real UD-Q4_K_XL GGUF (GB10, `--gpu`, keep-quant), same " Paris.…" text. `LagunaKvCache` (mirrors `DeepseekV4KvCache`, MLA-latent → GQA multi-head K/V; caches post-QK-RMSNorm/post-RoPE K + raw V at f32 — bit-exact since RoPE/QK-norm are position-only and attention is causal), MIXED attention per-layer: 12 GLOBAL layers grow unbounded + 36 SLIDING-WINDOW-512 layers EVICT rows beyond the 512 window (gemma2/3 `is_sliding`); `LagunaForwardGgufCached` + shared `LagunaAttention`/`LagunaFfnBlock` helpers used by BOTH forwards (identical float ops; recompute ids unchanged after refactor), `examples/laguna_gen --stateless` A/B flag. No cache bug (bit-exact first run). Next speed = grouped-expert GEMM + device-resident decode (both in-tree from ds4). See `.agents/specs/laguna-s21-w6-2026-07-31.md`. Prior RUNNABLE (W5, 2026-07-31): our engine greedy-generates COHERENT text on the REAL 3-shard UD-Q4_K_XL GGUF (GB10 keep-quant) — "The capital of France is" → " Paris. …", first token "Paris." matches the llama.cpp-Poolside reference. Multi-shard GGUF reader + keep-quant tower (`LoadLagunaFromGgufShards`) + `LagunaForwardGguf` (ds4 keep-quant Gemm/GemmRowSlice) + `examples/laguna_gen`; load 20.6s, peak 71 GiB, 3.27 s/tok stateless recompute (speed=W6).** Prior W3: **W3 REAL forward + 3 new ops landed** (`laguna_ops.cpp`: per-head softplus attn out-gate + ungrouped sigmoid-noaux router + dual per-layer RoPE cos/sin builders; `LagunaModel::Forward` now a REAL runnable host-reference composition — variable-Q-head GQA + dual RoPE + sliding-window mask + softplus gate + dense L0 / ungrouped-MoE L1..47 + untied lm_head — replacing the W1/W2 `VT_CHECK(false)` stub; CPU `-Werror` full-library build clean; `test_laguna_scaffold` **8/8·166** incl. softplus math, router selection+tie-break RED-first, dual-RoPE cos/sin bit-match, variable-Q-head shapes, forward composition on synthetic weights; `test_model_registry` 24/24). W1 oracle DECISION: vLLM native `laguna.py` in pin ⇒ config constructs; dual-oracle = vLLM-NVFP4/-FP8 (fits GB10, BF16 235 GiB does NOT) + llama.cpp-Q4_K token-exact. DEFERRED to W4 (needs 73 GB checkpoint): GGUF keep-quant tower materialization + device/paged production forward + strict dual-oracle greedy gate. ~85-90% reuse (ds4-MoE + gemma-sliding + olmo3-dual-rope + landed Q4_K keep-quant); NEW = the 3 landed host ops + name-map + variable-Q-head device runner. **W4 (2026-07-31, `CLAIM-LAGUNA-W4`, in progress):** the UD-Q4_K_XL GGUF (73.4 GiB, 3 shards) FETCHED to dgx + its metadata/tensor-map READ AUTHORITATIVELY (814 tensors, arch `laguna`, `expert_gating_func=2` sigmoid, `leading_dense_block_count=1`, `expert_weights_scale=2.5`). Three CPU-verified FIDELITY corrections the W1-W3 scaffold got wrong, each grounded in the real GGUF + llama.cpp: (1) **per-head QK-RMSNorm** (`attn_q_norm`/`attn_k_norm` F32[128]) added to params+forward — the scope MISSED it (surfaces only in the tensor map); (2) **dual-RoPE mscale** now uses llama.cpp's `yarn_attn_factor·(1+0.1·ln(factor))` off the GGUF-authoritative `factor=32`/`yarn_attn_factor=1.0` (256K-ctx build, NOT HF's factor-128/1.4852 1M-ctx scalar) — resolves the numerics-delicate residual; (3) **separate** `ffn_gate_exps`/`ffn_up_exps` (Q4_K) + `ffn_down_exps` (Q5_K) + Q8_0 shared/attn (the scaffold assumed merged gate_up). GGUF keep-quant tower materialization (`Mw`/`Sew` mirror of ds4) + keep-quant `ForwardGguf` (vt::MatmulBT/GemmRowSlice) + the real-model greedy run vs the llama.cpp-laguna same-quant oracle remain the W5 close (73 GB single-GB10, host-orchestrated) | `MODEL-TEXT-laguna-laguna-for-causal-lm` | | 🚫 | `DeepseekV3ForCausalLM` / `DeepseekV32ForCausalLM` | DeepSeek-V3 / V3.2 | HW-blocked (671B, ~642 GiB fp8 vs 119 GiB unified memory); V3.2 additionally DEP-blocked (DSA indexer) | `MODEL-TEXT-deepseek-v2-deepseek-v3-for-causal-lm` | diff --git a/.agents/specs/minimax-music3.md b/.agents/specs/minimax-music3.md index 510f5210e..b6c34a224 100644 --- a/.agents/specs/minimax-music3.md +++ b/.agents/specs/minimax-music3.md @@ -1092,3 +1092,193 @@ do; it is recorded here because the obvious first read of a slow run is "the language model is slow", and the language model is not the part that is slow — the LM's own weight load is 180 s of I/O and its forward is 12-14% of the AR profile. + +--- + +## 10. The parity sweep, the music-only server, and the weights record (#672) + +**Developer directive (2026-08-15):** parity on what upstream supports — +"we want to be a good reference" — usage docs for MiniMax-Music3, and in those +docs the models and weights used and supported, the way MiniMax-H3 already does +it. Then, mid-flight: **"we should allow to load only the music model"** and +**"we need to have an e2e test working"**. The first two lines are the scope; +the last two fixed two of its answers as requirements rather than judgements. + +### 10.1 The upstream surface, enumerated + +SGLang-Omni `748a0b43` at `sglang_omni/models/minimax_music3/` and the diffusers +PR at `c6da9936` were read field by field. What a user can set upstream, and +where each lands here: + +| upstream field | upstream default and anchor | here | +|---|---|---| +| `prompt` / `instructions` (the description) | required, `encoders.py:194-198`; SGLang `request_builders.py:104-106` | `description` (alias `prompt`) — **PARITY** | +| `lyrics` / `input` | required, `encoders.py:199-200`; `request_builders.py:103` | `lyrics` — **PARITY** | +| `audio_duration` | 60.0 s, `encoders.py:251-259` | `audio_duration` (alias `duration`) — **PARITY**, same default | +| `num_inference_steps` | 30, `denoise.py:141-148` | `num_inference_steps` — **PARITY**, same default | +| CFG scale | **not a request field** — frozen at 1.7 into the guider component, `denoise.py:180`; a serve-time knob `dit_cfg_scale` in SGLang, `stages.py:76-95` | `guidance_scale`, a real per-request control defaulting to 1.7. **AHEAD of both arms** | +| `generator` / `seed` | a `torch.Generator` in diffusers (`encoders.py:260`, `denoise.py:111`); an integer defaulting to 0 in SGLang (`payload_types.py:25`) | `seed`, integer, default 0 — **PARITY** with the SGLang spelling | +| `max_new_tokens` (frames) | 9000 cap, `request_builders.py:56-68` | **REFUSED BY NAME**, pointing at `audio_duration` and the /25 conversion | +| `temperature`, `top_p`, `top_k`, `repetition_penalty` | **refused** by upstream, `request_builders.py:14-19,109-114` | **REFUSED BY NAME** — was SILENT, and that silence was the #925 class | +| `voice`, `speed` | refused, `request_builders.py:83-92` | refused — **PARITY** | +| `stream` | refused, `request_builders.py:115-116`; `supports_streaming_vocoder=False` | refused — **PARITY**. Upstream has no streaming in either arm | +| `response_format` | wav/mp3/flac/pcm/aac/opus, `protocol.py:291` | `"wav"` only — **OWED**, no encoder is vendored. Note upstream **downmixes to mono** for any non-wav format (`client/audio.py:328-334`) | +| prompt ceiling 5000 tokens | `encoders.py:42,212-215` | enforced, `minimax_music3_ar.cpp:226` — **PARITY** | +| frame ceiling 9000 | diffusers **CLAMPS** silently (`encoders.py:287`); SGLang **REJECTS** (`request_builders.py:64-67`) | we CLAMP, mirroring the primary oracle. Gated at 360 s and 3600 s | +| output rate | diffusers 44100, no resample; SGLang resamples to 32000 (`acoustic.py:55-58,423`) | 44100 native — the §1.1 decision. The 32 kHz delivery transform stays **OWED** | +| N samples per request | **neither arm supports it** (`denoise.py:117-122` is batch 1; no `n` field on `protocol.py:334-368`) | one waveform per request — **PARITY** | +| N concurrent requests batched | SGLang only: continuous batching at 16, **two engine rows per request** for the CFG twin (`engine_builder.py:74-77`), plus `POST /v1/audio/speech/batch` (`openai_api.py:1277`) | we serialize per engine handle — **OWED** | +| `sgl-omni serve --model ` and nothing else | the norm: the pipeline is three stages with no chat LLM, `models/minimax_music3/config.py:29-63` | **CLOSED** — see §10.2 | + +**Closed by this change:** the music-only server, the missing example, the four +sampling refusals, the `max_new_tokens` refusal. +**Refused by name and recorded as owed:** the non-wav response formats, request +batching and the `/batch` route, the 32 kHz delivery resample, the native `.pth` +arm, streaming (which upstream does not have either, so it is a permanent +refusal rather than a debt). + +### 10.2 `--model` is optional when `--speech-model` is given + +Serving a 28.5 GB music model also forced loading an unrelated text model, +because `--model ` was unconditionally required. On this box the smallest +available text checkpoint is 35B, so **the recipe this project documented was +effectively unrunnable**, and upstream's own is `sgl-omni serve --model +MiniMaxAI/MiniMax-Music3` with no text tower anywhere. + +`--speech-model` alone now loads only the speech engine and registers only +`/v1/audio/speech`. It is the third instance of a shape already in +`server_main.cpp` — a pooling checkpoint serves `/v1/embeddings` alone, a +Parakeet checkpoint serves `/v1/audio/transcriptions` alone — and it mirrors +vLLM's task-conditional registration (`api_server.py:255-265`). + +**It is ADDITIVE and that is proved, not argued.** The only case whose verdict +changes is `--model` absent *and* `--speech-model` absent, which was an error +and remains one, with a message that now names both ways to satisfy it. +`--model` alone and `--model` + `--speech-model` take byte-identical paths. + +The route table is gated **in both directions over a real socket**, because a +handler-dispatch test cannot see route registration at all: with no synthesizer +`/v1/audio/speech` is a 404 from the route table with no envelope leaked, and on +a speech-only server `/v1/completions` and `/v1/chat/completions` are 404 while +`/v1/audio/speech` returns `audio/wav`. + +### 10.3 The e2e gate: what it examined, reported rather than implied + +The gate reported `test cases: 5 | 5 passed` and **`assertions: 0`** whenever the +checkpoint was absent. Five green case names over an empty run — the same shape +that fooled this project on `test_qwen3_paged_engine`, which "passes 2/2" while +asserting nothing because its snapshots are dgx-only. + +The file is now split. **The checkpoint-free half runs unconditionally in CI**: +the request contract on the exact body the real case posts, the near-miss +refusals, the duration arithmetic including both ceilings, and the speech-only +route table over a real socket with a stub synthesizer. `assertions: 0` is +therefore structurally impossible. **The checkpoint half** keeps its env gate, +and the real case now runs over a real socket against the music-only server +shape rather than calling `handle_audio_speech` directly. + +A **coverage-report case** prints, every run, which arms ran and why any did not. +Its assertion deliberately is **not** a cross-case counter: `-tc="…COVERAGE…"` +runs it alone, the counter is legitimately zero, and a gate that reds for the way +it was invoked is a gate somebody deletes. It asserts a cheap fact about the +checkpoint itself instead — 44100 Hz, hop 512, vocab 200000, 8 codebooks, read +from the component `config.json` files in milliseconds — which holds under any +invocation. + +**All three arms, measured on this box 2026-08-15**, so the difference between +them is visible rather than asserted: + +| arm | cases | assertions | what ran | +|---|---|---|---| +| no env vars | 9 | **37** | the checkpoint-free half only. Was 5 / **0** | +| `VLLM_CPP_MUSIC3_CHECKPOINT` | 9 | **86** | + decode, WAV and condition-mix; `checkpoint_arms_run=3` | +| + `VLLM_CPP_MUSIC3_DIT=1` | 9 | **582** | + the full tail and the music-only server over a real socket; `checkpoint_arms_run=5` | + +The full arm's own numbers: `POST /v1/audio/speech -> 200 audio/wav, 12332 bytes +in 518.0 s wall`; 2 AR frames -> 6 latent frames -> 3072 samples per channel +(0.0697 s); 6144 int16 samples, all non-zero, 0 clipped, 2818 of 3072 positions +differing between left and right; and `/v1/completions` and +`/v1/chat/completions` both 404 from the route table, which is the music-only +claim made over the wire against the real 28.5 GB engine rather than a stub. + +### 10.4 The weights are documented (porting-a-model.md §2.1) + +`docs/USAGE.md` carries the tables the H3 sections already carried, one row per +artifact, with the repo **and revision**: the diffusers arm at +`MiniMaxAI/MiniMax-Music3` @ `fbdf52fbaaca799592917417eb05f1899f1255ec`, +component by component, **28.5 GB resident** (28 517 617 303 B, measured) out of +a 57.4 GB repository and why the two differ; the native `.pth` arm we refuse and +that SGLang-Omni serves; the one implemented GGUF Q4_K artifact with its sha256; +and the fourteen third-party quantized repositories in five formats, each marked +refused and each marked third-party. + +The revision is **verified rather than copied**: +`condition_encoder/diffusion_pytorch_model.safetensors` on disk hashes to +`83179c5eaa9a68a370affe0c1b96c2179f659ea4175666b31071490a202c2a4d`, which is that +revision's own LFS record for the file. + +### 10.5 The first sample a human can hear, and where it is not + +**2.0 s of 44100 Hz stereo, from this engine, in 3286 s of wall clock.** The +e2e gate's own artifact is 0.07 s — the shortest request that still enters every +stage — which nobody can listen to. `minimax-music3-gen` at `--duration 2.0 +--steps 2 --seed 7` produced 88 064 frames per channel: RMS 0.03169, peak +0.97437 full-scale with **0 clipped samples**, 175 858 of 176 128 int16 samples +non-zero, and 84 073 of 88 064 positions differing between left and right, so +the 128 latent channels are folded into two streams of 64 rather than +interleaved. Verified independently of the generator, by re-reading the RIFF +file. + +x86 20-core CPU, load average swinging 7 to 150 across the run (several other +sessions on the box), 17.8 GB resident. No speed claim is made or implied: the +acoustic half is upstream's own fp32 and the depth decoder and DiT are scalar +host loops by construction (see `## Now`). + +**Its samples are compared to nothing, and that is structural rather than an +omission.** §5 withdrew the token gate; §6/W6 records that a request's waveform +can never equal `waveform.npy` because both the codes and the initial latents +are seeded random draws. The clip demonstrates the pipeline runs and emits a +well-formed, non-silent, non-clipped, genuinely stereo signal. The per-stage +gates are what speak to correctness. + +**It is NOT committed, and the reason is a checker rather than a preference.** +`scripts/check-pr-size.py` classifies every repository path; `ASSET` accepts +`assets/*.{png,svg}`, `BENCH_EVIDENCE` accepts +`benchmarks/{demo,media}/*.{json,png,gif,mp4,log}`, and neither takes a `.wav`. +The only classified home for one is under `tests/`, where a file compared to +nothing would sit beside the oracle goldens and imply it was one — which +`test_minimax_music3_e2e_real.cpp` explicitly refuses for its own artifact +("under the build tree, never under tests/ — no golden is created, replaced or +implied by this"). Widening either pattern would be widening a checker's scope +to make a change pass, which AGENTS.md forbids without its own spec and +red-before evidence, and this clip does not justify one. Regenerating it is one +command. + +### 10.6 A red that belonged to nobody, found by checking a matched arm (#965) + +`windows-msvc-cpu` and `windows-msvc-vulkan` failed on this row's pull request. +Both are habitually red and both are habitually attributed to +[#645](https://github.com/mudler/vllm.cpp/issues/645). **They were not #645.** +#645 is the `M_PI` portability regression in three LTX2 sources; this was: + +``` +server_main.cpp(1315,55): error C2220: the following warning is treated as an error +server_main.cpp(1315,55): warning C4456: declaration of 'loaded' hides previous local declaration +``` + +— W6's own speech-attach block declaring `loaded` inside the scope of the text +engine's `loaded` at `:1025`. The only warning in the job, and on `main` since +W6 landed. + +**What found it was the matched-arm check, not the label.** Three unrelated open +pull requests — #956, #950, #939, none touching the speech surface — fail with +the identical `C4456`. That is what separates "pre-existing" from "mine", and it +is the step that a known-red list invites you to skip. Because `windows-msvc-*` +are PR-only ([#584](https://github.com/mudler/vllm.cpp/issues/584)), `main` +carries no baseline, so the failure presents to every author in turn as a red +their own diff caused — and a second cause sitting behind a known one is +invisible for exactly as long as nobody reads the log. + +Fixed in flow: the inner declaration is renamed, with a comment saying why the +name is not `loaded`. No detector weakened, no warning suppressed, no behaviour +changed. diff --git a/docs/FEATURES.md b/docs/FEATURES.md index c71a9abde..9023d9670 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -198,7 +198,7 @@ on the committed fixture); reranking/classify models are not yet registered. | Audio | ✅ correctness-gated | ✅ | ◐ | ◐ | | Video+audio GENERATION (MiniMax-H3 DiT, LTX-2.5 DiT) | ◐ H3: all three modalities COHERENT on Q4_K_M (t2va, fl2va, ref2va; §8.20); the NVFP4 arm carries the patch grid; GGUF/NVFP4/bf16 loaders, pruned too (§8.21). LTX-2.5: a second lane, `SPIKE`, gated at reduced dims | ✅ H3 (vllm-omni, BF16-only, no quantized arm); LTX-2.5 only through the generic diffusers adapter, no native recipe ([vllm-omni#6066](https://github.com/vllm-project/vllm-omni/issues/6066)) | ☐ | ☐ | | Speech / audio GENERATION (TTS, vLLM-Omni lane) | ◐ IndexTTS-2.5: vllm_synthesize renders TEXT to AUDIO on real weights, but the reference clip is IGNORED and CAMPPlus returns NaN on real weights (#634, #633) | ✅ (vllm-omni: MOSS-TTS, Qwen3-TTS, Higgs Audio v3, Voxtral TTS, IndexTTS-2.5) | not assessed | not assessed | -| MUSIC generation (MiniMax-Music3) | ✓ every stage gated, and a composed HTTP request observed end to end on CPU: a 44100 Hz stereo WAV in 7:54 (#852, [spec](../.agents/specs/minimax-music3.md), #672) | ☐ absent from the pin, from vLLM `main` and from `vllm-omni` alike | ◐ served by SGLang-Omni, a third repository, which loads the NATIVE checkpoint layout | ☐ | +| MUSIC generation (MiniMax-Music3) | ✓ every stage gated; an HTTP request observed e2e over a REAL SOCKET against a MUSIC-ONLY server (#852, #672, [spec](../.agents/specs/minimax-music3.md) §10) | ☐ absent from the pin, from vLLM `main` and from `vllm-omni` | ◐ SGLang-Omni serves the NATIVE layout; its 32 kHz resample and batching are OWED | ☐ | | Multimodal over the OpenAI server | ◐ image request path wired, forward pending | ✅ | ✅ | ◐ | Image, video and audio are correct through the CLI and library. Over the HTTP diff --git a/docs/USAGE.md b/docs/USAGE.md index 54e44c857..d8ea4e776 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -1338,9 +1338,24 @@ identical to one built before it existed. See ### Speech and music generation +A **music-only server**, which is what you almost certainly want: + + vllm-server --speech-model /path/to/minimax-music3 \ + [--speech-family minimax-music3] [--port 8000] + +**`--model` is not required here**, and that is deliberate. Upstream's own recipe +is `sgl-omni serve --model MiniMaxAI/MiniMax-Music3` and nothing else: a music +model is not an accessory to a text model. With `--speech-model` alone this +server loads the music checkpoint, registers `/v1/audio/speech`, and registers +**nothing else** — no `/v1/completions`, no `/v1/chat/completions`. That is the +same task-conditional shape a pooling checkpoint (`/v1/embeddings` only) and a +Parakeet checkpoint (`/v1/audio/transcriptions` only) already take here, and the +same one vLLM's `api_server.py:255-265` uses. + +Attach it to a text server instead, and one process serves both surfaces: + vllm-server --model /path/to/text-model \ - --speech-model /path/to/minimax-music3 \ - [--speech-family minimax-music3] + --speech-model /path/to/minimax-music3 `--speech-model` names the checkpoint **set** — MiniMax-Music3 ships six component directories beside a `modular_model_index.json`, so this is not a @@ -1348,7 +1363,25 @@ single model directory. `--speech-family` is optional: omitted, the family is **detected** by inspecting the artifact, and a directory no registered family claims is refused at startup naming every family that was tried. A name that is not registered is refused too; it is never treated as a hint, because the wrong -family would not fail — it would render noise. +family would not fail — it would render noise. `--speech-family` without +`--speech-model` is still an error: there is nothing to load it from. + +In the speech-only form the served model name defaults to the **family** +(`minimax-music3`) rather than to a directory basename, because there is no +`config.json` to take one from. `--served-model-name` still wins. + +Or skip HTTP entirely. `minimax-music3-gen` drives the same seam through the C +ABI and writes the WAV itself: + + minimax-music3-gen --model /path/to/minimax-music3 --out song.wav \ + --lyrics @lyrics.txt --description "Genre: acoustic pop. BPM: 96." \ + --duration 8 --steps 8 --seed 7 + +`--lyrics` and `--description` take literal text or `@path` to read a file, +because lyrics are multi-line and a `[Verse]` tag inside an argv is easy to +mangle. It prints the delivered length, rate, channels, RMS, peak and wall clock +to stderr — the *delivered* length, not the requested one, because a duration +resolves to a whole number of 25 Hz frames and is therefore quantized. The route is OpenAI's `createSpeech` shape, with the two **music** inputs as additional named fields: @@ -1365,26 +1398,45 @@ The response body is RIFF/WAVE 16-bit PCM at the family's **native** rate (44100 Hz stereo for MiniMax-Music3, never resampled), with content type `audio/wav`. +**Every field, and what it does.** Anything not in this table is refused by name +rather than ignored — see below the table for why that polarity matters here. + +| field | type | default | what it does | +|---|---|---|---| +| `lyrics` | string | **required** for MiniMax-Music3 | the sung text, with `[Verse]` / `[Chorus]` section tags. An empty lyric normalizes to a bare `[start]` prompt, so it is a 400 rather than an instrumental | +| `description` (alias `prompt`) | string | **required** for MiniMax-Music3 | genre, BPM, key, instrumentation, mood. NOT a voice or speaker description. Supplying both spellings with different values is a 400, never a silent winner | +| `audio_duration` (alias `duration`) | number, seconds | `60` | resolved to `int(seconds x 25)` autoregressive frames, then **clamped** to the 9000-frame ceiling — the same silent clamp upstream applies (`encoders.py:287`). Shorter than one frame (0.04 s) is a 400 | +| `num_inference_steps` | integer | `30` | flow-matching Euler steps in the acoustic half. Must be > 0 | +| `guidance_scale` | number | `1.7` | classifier-free guidance on the DiT. **0 is legal** and selects the unconditional branch, so omitting the field is how you ask for the default — not sending 0 | +| `seed` | integer | `0` | seeds the autoregressive top-k draw *and* the initial denoise latents. A fixed seed, not a random one: 0 is as deterministic as any other value | +| `model` | string | — | echoed; the route does not check it | +| `response_format` | string | `"wav"` | `"wav"` is the only accepted value | + `lyrics` and `description` are separate fields rather than one `input` behind a -separator because upstream runs a different normalizer over each. A -one-utterance family keeps using OpenAI's `input`. `prompt` is the documented -alias for `description`, and supplying both with different values is a 400 -rather than a silent winner. - -The duration key is `audio_duration`, in seconds, with `duration` accepted as an -alias. Omit it and the family's own default applies, which is 60 s for -MiniMax-Music3. **`audio_duration_s` is refused**: that is the name of the field -the key fills, not a key, and accepting it would return the default duration -behind a 200 with nothing to tell the caller its request had been dropped. That -is not hypothetical, it cost this project's own end-to-end gate four multi-hour -runs, because 0.1 s silently became 60 s. - -Refused by name rather than ignored, because honouring any of them silently -would return audio the caller did not ask for: `voice` (no registered family -exposes named voices), `speed` (no family implements a rate control), `stream` / -`stream_format` (MiniMax-Music3 generates the whole song before the first sample -exists, so buffering it would be a stream in name only) and any -`response_format` other than `"wav"` (no mp3/opus/aac/flac encoder is vendored). +separator because upstream runs a different normalizer over each — `_clean_caption` +on the description, `_normalize_lyrics` on the lyrics (`encoders.py:54-91`). A +one-utterance family keeps using OpenAI's `input`; MiniMax-Music3 refuses it, so +a request cannot half-arrive. + +**We expose `guidance_scale` where neither upstream arm does.** In diffusers it +is frozen into the guider component at 1.7 (`denoise.py:180`); in SGLang-Omni it +is a serve-time knob (`dit_cfg_scale`) and not a request field. It is a genuine +per-request control here, and its default is upstream's 1.7. + +**Every refusal, and the one rule behind them.** A knob the server will not +honour must not come back behind a 200. Silently dropping one returns audio the +caller did not ask for with nothing to say so — and this project has already paid +for that once (#925), which is why the list is long rather than convenient. + +| refused | why | +|---|---| +| `audio_duration_s` | the name of the *field* the key fills, not a key. It is the misspelling you reach for by reading the struct instead of the docs, and dropping it silently returned the 60 s default: 0.1 s became 60 s, 2 autoregressive frames became 1500, and this project's own e2e gate spent four multi-hour runs inside a 750x job it read as a hung weight load (#852, #925) | +| `voice` | no registered family exposes named voices, and there is no enumeration endpoint to pick one from. Upstream refuses it too (`request_builders.py:90-92`) | +| `speed` | no family implements a rate control. Upstream accepts only `1.0` (`request_builders.py:83-89`) | +| `stream`, `stream_format` | MiniMax-Music3 generates the whole song before the first sample exists, so buffering it into chunks would be a stream in name only. **Upstream has no streaming either** — SGLang-Omni declares `supports_streaming_vocoder=False` and rejects `stream=true` by name (`request_builders.py:115-116`) | +| `response_format` other than `"wav"` | no mp3/opus/aac/flac encoder is vendored, and relabelling RIFF bytes is worse than refusing | +| `temperature`, `top_p`, `top_k`, `repetition_penalty` | this model's autoregressive stage has ONE sampler — a fixed top-50 draw (`encoders.py:48,94-103`). There is no temperature to set and no nucleus branch to widen, so the knob can be neither honoured nor honestly ignored. Upstream refuses all four (`request_builders.py:14-19,109-114`). Use `seed` to control the draw | +| `max_new_tokens` | SGLang-Omni's spelling of the length, counted in 25 Hz **frames** rather than seconds (`request_builders.py:56-68`). This route takes `audio_duration` in seconds — divide by 25. Two spellings of one meaning on one route is exactly what #925 was | A family with no text-only synthesis — IndexTTS-2.5 is one — is refused **before** anything stages: the route asks the loaded engine's @@ -1395,10 +1447,26 @@ WAV. **Every stage of MiniMax-Music3 is implemented and gated**, and a request reaches all of them: the 8.6B `Qwen3ForCausalLM` autoregressive stage, the RVQ depth decoder, the learned condition mix, the flow-matching DiT and the DAC -Flow-VAE vocoder. **A composed request has not yet been observed to completion -on CPU** — see the caveat below, and `.agents/specs/minimax-music3.md`. There is -no by-name refusal left: nothing here is unimplemented. IndexTTS-2.5 still -refuses naming its own missing pieces. +Flow-VAE vocoder. **A composed request has been observed to completion** — an +HTTP POST returns a real 44100 Hz stereo WAV (#852) — and the end-to-end gate +now runs it over a real socket against a music-only server. There is no by-name +refusal left: nothing here is unimplemented. IndexTTS-2.5 still refuses naming +its own missing pieces. + +**What no gate compares is the music itself.** The autoregressive codes are a +seeded `torch.multinomial` draw and the denoise loop's initial latents are a +seeded normal draw, so a request's waveform can never equal the oracle's golden +— twice over, and structurally rather than by omission. Every *stage* is gated +against the capture on the capture's own recorded inputs; a **generated** song +is evidence that the pipeline runs, not that the notes are right. Believe the +stage gates, and listen with that in mind. + +**Ask for less than 8 seconds while you are exploring.** `Music3ChunkPlan` only +splits past 200 autoregressive frames, which is 8 s of audio, and the +multi-window composition — the overlap blend, the carry span, the waveform crop +across windows — is this row's one named coverage gap: each primitive is gated +individually, the composition across windows is not, because the oracle capture +is a single 25-frame window. **It runs on CPU and it is slow.** Every gate this row has was taken on CPU (`dgx.casa` was down throughout), and the acoustic half is upstream's own fp32. @@ -1413,6 +1481,31 @@ that way in W2-W5 so their reduction order is reproducible against torch, and they run single-threaded. In one 0.1 s request the depth decoder alone is the majority of the wall clock. +**A first sample, measured.** Two seconds of stereo music, generated by this +engine through `minimax-music3-gen` on an idle-to-busy 20-core x86 CPU box: + +| property | value | +|---|---| +| duration | 1.9969 s (88 064 frames per channel) | +| rate / channels | 44 100 Hz, 2 channels, 16-bit PCM | +| RMS | 0.03169 full-scale | +| peak | 0.97437 full-scale, **0 clipped samples** | +| L != R | 84 073 of 88 064 positions, so the stereo fold is real rather than a duplicated channel | +| wall clock | **3286 s** (54.8 min) for 2.0 s of audio, at `--steps 2`, load average 40-150 throughout | + +**Its samples are compared to nothing.** The token gate this row once promised +was withdrawn — upstream's autoregressive stage has no greedy path — and a +generated waveform can never equal the oracle's golden anyway, because both the +codes and the initial latents are seeded random draws. So the numbers above +demonstrate that the pipeline runs end to end and produces a well-formed, +non-silent, non-clipped, genuinely stereo signal. They say nothing about whether +the music is right. The per-stage gates are what say that. + +The clip is **not committed**: `scripts/check-pr-size.py` classifies every +repository path, and no classified path accepts a `.wav` outside `tests/`, where +a file compared to nothing would sit beside the goldens and imply it was one. +Regenerate it instead — the command above is the whole recipe. + The same seam is reachable from the C ABI at v20 — `vllm_speech_engine_load`, `vllm_speech_engine_family` / `_sample_rate` / `_requires_reference_audio`, `vllm_synthesize` and `vllm_speech_result_free` — so HTTP and FFI drive one @@ -2642,6 +2735,112 @@ still says the projection is biased. Without the refusal that renders a plausibl video for the wrong prompt: every conditioning row is shifted by the missing bias and every padded row projects to 0 instead of to the bias. +## MiniMax-Music3: the exact weights (so a song is reproducible) + +**The repository is 57.4 GB and the arm we load is 28.5 GB**, because +`MiniMaxAI/MiniMax-Music3` ships the same weights **twice**: a native +`AbabForCausalLM` + `.pth` layout that SGLang-Omni serves, and a `diffusers` +six-component layout. They are the same numbers in a different arrangement — +diffusers' own `scripts/convert_minimax_music3_to_diffusers.py` renames tensors +and does nothing else — and this port loads the diffusers one. So the download +is 57.4 GB unless you filter, and what has to fit is 28.5 GB. + +### The arm that loads: `diffusers`, bf16 + fp32 + +Repository [MiniMaxAI/MiniMax-Music3](https://huggingface.co/MiniMaxAI/MiniMax-Music3), +revision **`fbdf52fbaaca799592917417eb05f1899f1255ec`**. First-party. A repo id +alone is not a pin — checkpoints do get re-quantized in place under an unchanged +name — so the revision is recorded, and it was verified rather than copied: +`condition_encoder/diffusion_pytorch_model.safetensors` on disk here hashes to +`83179c5eaa9a68a370affe0c1b96c2179f659ea4175666b31071490a202c2a4d`, which is +that revision's own LFS record for the file. + +| component | file(s) | size | dtype on disk | +|---|---|---|---| +| `language_model/` | `model-0000{1,2,3,4}-of-00004.safetensors` + index | **17.17 GB** | BF16 | +| `transformer/` | `diffusion_pytorch_model-0000{1,2}-of-00002.safetensors` + index | **9.73 GB** | **F32** | +| `rvq_depth_decoder/` | `diffusion_pytorch_model.safetensors` | **1.29 GB** | BF16 | +| `vocoder/` | `diffusion_pytorch_model.safetensors` | **217 MB** | F32 | +| `condition_encoder/` | `diffusion_pytorch_model.safetensors` | **101 MB** | F32 | +| `tokenizer/` | `tokenizer.json` + `tokenizer_config.json` + `chat_template.jinja` | **11 MB** | — | +| `scheduler/` | `scheduler_config.json` | 483 B | — | +| the root itself | `modular_model_index.json`, `config.json`, `README.md` | 14 KB | — | +| | **resident total** | **28.5 GB** (28 517 617 303 B) | | + +The transformer being 9.73 GB for a 2.4B model is **fp32 storage, not a 4.9B +model** — that is upstream's own choice for the acoustic half and we mirror it. +The download: + + hf download MiniMaxAI/MiniMax-Music3 --revision fbdf52fb \ + --local-dir "$CHECKPOINT_ROOT/minimax-music3" \ + --exclude 'qwen_7B/*' '*.pth' + +Two components are BF16 and three are F32, and **that set is not runnable as +stored**. Upstream casts in exactly two places, so the language model, the RVQ +depth decoder and the condition encoder must share one dtype; the gated +configuration is bf16 for those three and fp32 for the transformer and vocoder. +The loader enforces it and refuses a violation by name. The section below has +the detail. + +### The arm that is REFUSED: the native `.pth` layout + +The same repository's other 28.9 GB. **We refuse it by name** — a tree in this +shape is diagnosed as the native arm, told which diffusers components it lacks, +and pointed at the conversion script. It is never silently mis-loaded. + +| file | size | what it holds | +|---|---|---| +| `qwen_7B/qwen_7B/` | ~17 GB | `AbabForCausalLM` shards; the RVQ depth decoder and the audio embedding live *inside* them as `model.audio_decoder.*` / `model.audio_extra_embedding` | +| `flowmatching_vae.pth` | ~9.7 GB | the DiT plus the condition projection | +| `dav.pth` | ~0.2 GB | the DAC Flow-VAE decoder | + +**SGLang-Omni serves this arm exclusively.** If you are comparing against +`sgl-omni serve`, that is the layout it reads — same weights, so the comparison +is valid, but not the same files. + +### The quantized arm that IS implemented: GGUF Q4_K, one component + +| field | value | +|---|---| +| repo | [audio-cpp/MiniMax-Music3-GGUF](https://huggingface.co/audio-cpp/MiniMax-Music3-GGUF) — **third party**, not MiniMaxAI | +| revision | `c36aaeed683f33b05796788e4204f4eeba8fa547` | +| file | `rvq_depth_decoder_q4_k.gguf` | +| size | 405 752 480 bytes (406 MB, against 1.29 GB bf16) | +| sha256 | `4c5d41b27418d9c1046345f649cb61d7cde0e3bbda4af7f7cb142df2c70cbdd0` | +| contents | 47 tensors: 36 Q4_K projections, 9 BF16 norms, 2 F16 embedding tables | + +It is the **only** quantized arm implemented, and one component is not a +quantized model. The remaining four are refused by name and owed; the section +"MiniMax-Music3: the quantized arms" below records what each refusal says. + +### The quantized arms that are REFUSED — and they are all third-party + +**MiniMaxAI ships bf16/fp32 only.** A HuggingFace survey on 2026-08-14 found +**fourteen community repositories in five formats**, published within days of the +release, and none of them is from the model's authors. Every one carries +different provenance from a first-party release, and every one except the single +Q4_K file above is refused by name. + +| format | repositories | coverage | state | +|---|---|---|---| +| GGUF, `audiocpp` lineage | [audio-cpp/MiniMax-Music3-GGUF](https://huggingface.co/audio-cpp/MiniMax-Music3-GGUF) | all five components, bf16 and Q4_K arms | `rvq_depth_decoder_q4_k` **LOADS**; `transformer_q4_k` (1 396 MB), `language_model_q4_k` (7 184 MB), `vocoder` (217 MB) and `condition_encoder` (101 MB) are **OWED**. Note the last two are bf16 GGUF, not k-quant — same size as the safetensors, so they buy nothing | +| GGUF, `mm3` lineage | [scragnog/MiniMax-Music3-GGUF](https://huggingface.co/scragnog/MiniMax-Music3-GGUF) | 2-file split (`mm3-lm-*` / `mm3-synth-*`), 13 tiers incl. MXFP4 and NVFP4 as GGML tensor types | **REFUSED**: needs a rename table *plus* fused QKV to split and folded weight-norm to invert. Its NVFP4 tier uses GGML type id 40, which is not a standard llama.cpp id | +| GGUF, ComfyUI lineage | [Abiray](https://huggingface.co/Abiray), [realrebelai/MiniMax-Music-3_GGUFs](https://huggingface.co/realrebelai/MiniMax-Music-3_GGUFs), [molbal](https://huggingface.co/molbal), [ChrisColeTech](https://huggingface.co/ChrisColeTech) | the 2.46B **DiT alone**, Q2_K…Q8_0, 0.9-2.7 GB | **REFUSED, and it can never be a complete arm**: these files carry the DiT and condition encoder only — no language model, no depth decoder, no vocoder — so even a finished GGUF arm would not make them generate audio | +| int8 / w4a8 | [Comfy-Org/MiniMax-Music-3](https://huggingface.co/Comfy-Org/MiniMax-Music-3) (`_int8_convrot`), [NidAll/MiniMax-Music3-W4A8](https://huggingface.co/NidAll/MiniMax-Music3-W4A8), [dummy9996/…-w4a8-bf16-comfyui](https://huggingface.co/dummy9996) | DiT | **REFUSED** by name | +| MLX 4/6/8-bit | [ddalcu](https://huggingface.co/ddalcu), [vanch007](https://huggingface.co/vanch007), [elishabjm](https://huggingface.co/elishabjm) | | **REFUSED**: MLX is a shared seam this project implements for no model, so it is not a per-model addition | +| proprietary | [infosave/MiniMax-Music-3-cmf](https://huggingface.co/infosave/MiniMax-Music-3-cmf) (Cortiq 4-bit) | | **not implementable**, recorded rather than owed | + +**"The GGUF arm" is three mutually incompatible lineages, and +`general.architecture` cannot separate them** — it reads `audiocpp`, `mm3`, +`qwen3` and `wan` across files of the same model, and `wan` collides with genuine +Wan video GGUFs. That is why the detector keys on +`audiocpp.model_spec.family` instead, and why pointing a `.gguf` at this loader +gets a refusal naming the lineage rather than a shape error. + +**NOT found** by those queries on that date: AWQ, GPTQ, compressed-tensors, fp8 / +`fp8_e4m3fn` / `fp8_scaled`, bitsandbytes. That is "not found by these queries on +this date", never "does not exist". + ## MiniMax-Music3: the checkpoint loader **It loads, it does not generate.** `include/vllm/model_executor/models/` diff --git a/examples/CMakeLists.txt b/examples/CMakeLists.txt index f351412b9..ce863e24f 100644 --- a/examples/CMakeLists.txt +++ b/examples/CMakeLists.txt @@ -230,6 +230,42 @@ if(VLLM_CPP_SERVER) --video-extra dit_config_path) set_tests_properties(test_server_video_extra_needs_a_value PROPERTIES PASS_REGULAR_EXPRESSION "--video-extra takes KEY=VALUE, got 'dit_config_path'") + + # A SPEECH-ONLY server (#672): --speech-model with NO --model. Upstream's own + # recipe is `sgl-omni serve --model MiniMaxAI/MiniMax-Music3` and nothing + # else, and requiring --model here meant serving a 28.5 GB music model also + # staged an unrelated text model no request would touch. Black-box for the + # same reason the --video-family cases above are: the flag table lives in an + # anonymous namespace and the binary IS the surface. + # + # 1. NEITHER flag is still an error, and the message now names the second + # way to satisfy it. The pre-change spelling had no --speech-model + # clause, so this expression is the RED. + add_test(NAME test_server_neither_model_flag + COMMAND $) + set_tests_properties(test_server_neither_model_flag PROPERTIES + PASS_REGULAR_EXPRESSION "--model is required \\(or --speech-model for a speech/music-only server\\)") + + # 2. --speech-model ALONE gets PAST the argument check and into the speech + # LOAD, which is the whole claim: before the change it exited at + # "--model is required" without ever reaching a checkpoint. Proven + # by the load refusal for a directory nothing claims, which is evidence + # that the speech branch ran, and by asserting the --model demand is + # ABSENT from the output. + add_test(NAME test_server_speech_only_reaches_the_speech_load + COMMAND $ --speech-model /nonexistent-speech-dir) + set_tests_properties(test_server_speech_only_reaches_the_speech_load PROPERTIES + PASS_REGULAR_EXPRESSION "--speech-model /nonexistent-speech-dir" + FAIL_REGULAR_EXPRESSION "--model is required") + + # 3. --speech-family STILL requires --speech-model. The new accepted + # combination must not have widened this one into "a family with no + # checkpoint is fine now". + add_test(NAME test_server_speech_family_still_needs_a_checkpoint + COMMAND $ --model /nonexistent-model-dir + --speech-family minimax-music3) + set_tests_properties(test_server_speech_family_still_needs_a_checkpoint PROPERTIES + PASS_REGULAR_EXPRESSION "--speech-family names a family but --speech-model names no checkpoint") endif() endif() @@ -302,3 +338,12 @@ endif() add_executable(parakeet-transcribe parakeet_transcribe/main.cpp) target_link_libraries(parakeet-transcribe PRIVATE vllm::shared) vllm_cpp_set_warnings(parakeet-transcribe) + +# minimax-music3-gen: MiniMax-Music3 text-to-music end to end — a THIN client of +# the public C ABI (include/vllm.h + vllm::shared ONLY, like vllm-cli and +# parakeet-transcribe). It exists because the music family was the one +# generative family in this tree with NO example: hearing the model needed a +# running server plus a curl, or a C ABI caller nobody had written (#672). +add_executable(minimax-music3-gen minimax_music3_gen/main.cpp) +target_link_libraries(minimax-music3-gen PRIVATE vllm::shared) +vllm_cpp_set_warnings(minimax-music3-gen) diff --git a/examples/minimax_music3_gen/main.cpp b/examples/minimax_music3_gen/main.cpp new file mode 100644 index 000000000..53ee68e67 --- /dev/null +++ b/examples/minimax_music3_gen/main.cpp @@ -0,0 +1,247 @@ +// minimax-music3-gen — MiniMax-Music3 text-to-music end to end, as a THIN +// CLIENT of the public C ABI (include/vllm.h) and NOTHING else, per the ONE +// SURFACE directive. It includes no internal header, and every capability it +// reaches is one an embedder gets: vllm_speech_engine_load, the three +// interrogations of the loaded handle, vllm_synthesize, and the RIFF bytes the +// result already carries. +// +// It exists because the model had NO example. `/v1/audio/speech` needed a +// running server plus a `curl`, and the C ABI needed a caller nobody had +// written — so the shortest path to hearing this model was neither. Every other +// generative family in this tree has one (minimax-h3-gen, ltx2-gen), and the +// music family was the exception. +// +// minimax-music3-gen --model --out +// --lyrics [--description ] +// [--duration SECONDS] [--steps N] [--guidance F] +// [--seed N] [--family ] +// +// `--model` names the checkpoint SET, not a file: the diffusers arm ships six +// component directories beside a `modular_model_index.json`. `--family` is +// optional — omitted, the family is DETECTED by inspecting the artifact, and a +// directory no registered family claims is refused naming every family tried. +// +// `--lyrics` and `--description` take either literal text or `@path` to read a +// file, because lyrics are multi-line by nature and a shell heredoc inside an +// argv is how a `[Verse]` tag ends up mangled. +// +// WHAT THIS CANNOT DO, refused rather than faked: there is no streaming (the +// whole song exists before the first sample does — upstream has none either), +// no format other than RIFF/WAVE 16-bit PCM (no mp3/opus encoder is vendored), +// and no resample off the family's native 44100 Hz stereo. Those are the same +// refusals `/v1/audio/speech` makes, for the same reasons. +// +// IT IS SLOW ON CPU and no speed number is claimed. The acoustic half is +// upstream's own fp32 and the AR half's host GEMM is a scalar loop written for +// a reproducible reduction order (.agents/specs/minimax-music3.md `## Now`). +// Ask for a short `--duration` and few `--steps` while checking that it works. +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vllm.h" + +namespace { + +// `@path` reads the file; anything else is the literal text. Returns false only +// when a `@path` cannot be read — a literal never fails. +bool ResolveText(const std::string& arg, std::string* out) { + if (arg.empty() || arg[0] != '@') { + *out = arg; + return true; + } + const std::string path = arg.substr(1); + std::FILE* f = std::fopen(path.c_str(), "rb"); + if (f == nullptr) { + std::fprintf(stderr, "error: cannot read %s\n", path.c_str()); + return false; + } + out->clear(); + char buf[4096]; + size_t n = 0; + while ((n = std::fread(buf, 1, sizeof(buf), f)) > 0) out->append(buf, n); + std::fclose(f); + return true; +} + +void Usage(const char* argv0) { + std::fprintf(stderr, + "usage: %s --model --out \n" + " --lyrics [--description ]\n" + " [--duration SECONDS] [--steps N] [--guidance F]\n" + " [--seed N] [--family ]\n" + "\n" + " --model the checkpoint SET directory (six component dirs +\n" + " modular_model_index.json), NOT a single file\n" + " --lyrics the sung text, with [Verse]/[Chorus] section tags.\n" + " REQUIRED: there is nothing to sing without it\n" + " --description genre, BPM, key, instrumentation, mood. NOT a voice\n" + " --duration seconds of audio; omitted => the family default (60 s)\n" + " --steps denoise steps; omitted => the family default (30)\n" + " --guidance CFG scale; 0 IS legal, so omitted (not 0) means default\n" + " --seed seeds the AR sampling and the initial denoise latents\n" + " --family skip detection and name the family\n", + argv0); +} + +} // namespace + +int main(int argc, char** argv) { + std::string model, out_path, family; + std::string lyrics_arg, description_arg; + double duration = 0.0; // <= 0 => family default + int64_t steps = 0; // <= 0 => family default + double guidance = 0.0; // only honoured when guidance_given + bool guidance_given = false; // 0 is a LEGAL guidance scale, so presence is a flag + int64_t seed = 0; + + for (int i = 1; i < argc; ++i) { + const std::string flag = argv[i]; + auto next = [&](const char* name) -> std::string { + if (i + 1 >= argc) { + std::fprintf(stderr, "error: %s takes a value\n", name); + Usage(argv[0]); + std::exit(2); + } + return argv[++i]; + }; + if (flag == "--model") { + model = next("--model"); + } else if (flag == "--out") { + out_path = next("--out"); + } else if (flag == "--lyrics") { + lyrics_arg = next("--lyrics"); + } else if (flag == "--description" || flag == "--prompt") { + description_arg = next(flag.c_str()); + } else if (flag == "--duration") { + duration = std::atof(next("--duration").c_str()); + } else if (flag == "--steps") { + steps = std::atoll(next("--steps").c_str()); + } else if (flag == "--guidance") { + guidance = std::atof(next("--guidance").c_str()); + guidance_given = true; + } else if (flag == "--seed") { + seed = std::atoll(next("--seed").c_str()); + } else if (flag == "--family") { + family = next("--family"); + } else if (flag == "--help" || flag == "-h") { + Usage(argv[0]); + return 0; + } else { + std::fprintf(stderr, "error: unknown argument '%s'\n", flag.c_str()); + Usage(argv[0]); + return 2; + } + } + if (model.empty() || out_path.empty() || lyrics_arg.empty()) { + std::fprintf(stderr, "error: --model, --out and --lyrics are required\n"); + Usage(argv[0]); + return 2; + } + + std::string lyrics, description; + if (!ResolveText(lyrics_arg, &lyrics)) return 1; + if (!description_arg.empty() && !ResolveText(description_arg, &description)) return 1; + + vllm_speech_model_params mp = vllm_speech_model_params_default(); + mp.path = model.c_str(); + if (!family.empty()) mp.family = family.c_str(); + + const auto load_t0 = std::chrono::steady_clock::now(); + vllm_speech_engine* engine = nullptr; + if (vllm_speech_engine_load(&mp, &engine) != VLLM_OK) { + std::fprintf(stderr, "load failed: %s\n", vllm_last_error()); + return 1; + } + const double load_s = + std::chrono::duration(std::chrono::steady_clock::now() - load_t0).count(); + std::fprintf(stderr, "loaded family '%s' at %d Hz in %.1f s\n", + vllm_speech_engine_family(engine), vllm_speech_engine_sample_rate(engine), + load_s); + + // Ask BEFORE synthesizing, which is the whole reason this interrogation is on + // the ABI: a family with no text-only synthesis is a caller-side refusal + // rather than a job that fails after staging tens of gigabytes. + if (vllm_speech_engine_requires_reference_audio(engine) != 0) { + std::fprintf(stderr, + "error: family '%s' requires a reference clip, which this example does not " + "supply — it drives the text-only music path\n", + vllm_speech_engine_family(engine)); + vllm_speech_engine_free(engine); + return 1; + } + + vllm_speech_params sp = vllm_speech_params_default(); + sp.lyrics = lyrics.c_str(); + if (!description.empty()) sp.description = description.c_str(); + sp.audio_duration_s = duration; + sp.num_inference_steps = static_cast(steps); + if (guidance_given) { + sp.guidance_scale = guidance; + sp.has_guidance_scale = 1; + } + sp.seed = seed; + + const auto gen_t0 = std::chrono::steady_clock::now(); + vllm_speech_result result; + const vllm_status st = vllm_synthesize(engine, &sp, &result); + const double gen_s = + std::chrono::duration(std::chrono::steady_clock::now() - gen_t0).count(); + if (st != VLLM_OK) { + std::fprintf(stderr, "synthesize failed: %s\n", vllm_last_error()); + vllm_speech_engine_free(engine); + return 1; + } + + // The result carries the RIFF bytes as well as the float samples, so writing a + // playable file needs no second encoder here. + std::FILE* sink = std::fopen(out_path.c_str(), "wb"); + if (sink == nullptr) { + std::fprintf(stderr, "error: cannot write %s\n", out_path.c_str()); + vllm_speech_result_free(&result); + vllm_speech_engine_free(engine); + return 1; + } + const size_t written = + std::fwrite(result.wav, 1, static_cast(result.n_wav), sink); + std::fclose(sink); + if (written != static_cast(result.n_wav)) { + std::fprintf(stderr, "error: short write to %s (%zu of %lld bytes)\n", out_path.c_str(), + written, static_cast(result.n_wav)); + vllm_speech_result_free(&result); + vllm_speech_engine_free(engine); + return 1; + } + + // Report what came back rather than what was asked for. A duration is a + // REQUEST: it resolves to a whole number of 25 Hz autoregressive frames and + // then to a whole number of latent frames, so the delivered length is + // quantized and printing the request instead would misreport the file. + const double seconds = result.sample_rate > 0 + ? static_cast(result.n_samples) / result.sample_rate + : 0.0; + double peak = 0.0; + double energy = 0.0; + const int64_t total = result.n_samples * result.channels; + for (int64_t i = 0; i < total; ++i) { + const double v = result.samples[i]; + const double a = v < 0.0 ? -v : v; + if (a > peak) peak = a; + energy += v * v; + } + const double rms = total > 0 ? std::sqrt(energy / static_cast(total)) : 0.0; + std::fprintf(stderr, + "wrote %s: %.3f s, %d Hz, %d channel(s), %lld samples/channel, " + "RMS %.5f, peak %.5f, %.1f s wall\n", + out_path.c_str(), seconds, result.sample_rate, result.channels, + static_cast(result.n_samples), rms, peak, gen_s); + + vllm_speech_result_free(&result); + vllm_speech_engine_free(engine); + return 0; +} diff --git a/src/vllm/entrypoints/openai/server_main.cpp b/src/vllm/entrypoints/openai/server_main.cpp index 552a843fd..868fcb7a8 100644 --- a/src/vllm/entrypoints/openai/server_main.cpp +++ b/src/vllm/entrypoints/openai/server_main.cpp @@ -400,6 +400,8 @@ const InertArg* FindAcceptedInertArg(const std::string& flag) { " [--speech-model ] " "[--speech-family ]\n" " [--version]\n" + " --speech-model WITHOUT --model serves /v1/audio/speech ALONE (no " + "text model is loaded)\n" " accepted for published-recipe compatibility, NO effect: " "--enable-auto-tool-choice, --trust-remote-code\n"; std::exit(code); @@ -640,8 +642,22 @@ Args ParseArgs(int argc, char** argv) { Usage(argv[0], 2); } } - if (a.model_dir.empty()) { - std::cerr << "server: --model is required\n"; + // --model is required EXCEPT for a speech-only server (#672). Upstream's own + // spelling for MiniMax-Music3 is `sgl-omni serve --model MiniMaxAI/MiniMax-Music3` + // and nothing else — there is no text model in that command, because a music + // model is not an accessory to one. Requiring --model here made the documented + // recipe unrunnable on any box whose smallest text checkpoint is tens of GB: + // serving a 28.5 GB music model also staged a model no request would touch. + // + // This is a NEW ACCEPTED COMBINATION, not a change of behaviour: `--model` with + // or without `--speech-model` resolves exactly as before, and the case that + // changes verdict — NEITHER flag — was and remains an error. It mirrors the + // task-conditional dispatch already in this file for pooling and + // transcription-only models (vLLM api_server.py:255-265): the routes a server + // registers follow from what it loaded. + if (a.model_dir.empty() && a.speech_model.empty()) { + std::cerr << "server: --model is required (or --speech-model for " + "a speech/music-only server)\n"; Usage(argv[0], 2); } if (a.max_num_seqs <= 0 || a.max_num_batched_tokens < 0 || @@ -796,6 +812,67 @@ int VllmServerMain(int argc, char** argv) { std::cerr << "\n"; } + // ── SPEECH TASK DISPATCH (#672): --speech-model with NO --model serves + // /v1/audio/speech and nothing else. It is the same task-conditional shape + // the pooling and transcription-only branches below take, and the same one + // vLLM's api_server.py:255-265 uses: a server registers the routes its + // loaded task can answer, rather than every route it knows how to spell. + // + // It exists because upstream's own recipe is `sgl-omni serve --model + // MiniMaxAI/MiniMax-Music3` — one model, no text tower — and because pairing + // a 28.5 GB music checkpoint with an unrelated text checkpoint nobody + // queries is not a smaller cost than the music model itself. NOTHING here + // touches the combined path: --model + --speech-model still loads both and + // registers both, byte for byte as before. + if (args.model_dir.empty()) { + vllm::multimodal::SpeechRegistry& registry = vllm::multimodal::GlobalSpeechRegistry(); + vllm::models::music3::RegisterBuiltinSpeechFamilies(registry); + vllm::multimodal::SpeechModelParams smp; + smp.path = args.speech_model; + smp.family = args.speech_family; // empty => DETECT by inspecting the artifact + std::string why; + std::unique_ptr loaded_speech = registry.Load(smp, &why); + if (loaded_speech == nullptr) { + throw std::runtime_error("server: --speech-model " + args.speech_model + ": " + why); + } + std::shared_ptr speech_only(std::move(loaded_speech)); + // The served name defaults to the FAMILY rather than to the directory + // basename, because a speech-only server has no config.json to name and + // the family is what a client puts in `"model"`. An explicit + // --served-model-name still wins. + const std::string speech_served_name = + args.served_model_name.empty() ? speech_only->family() : args.served_model_name; + vllm::openai::SpeechCapabilities caps; + caps.family = speech_only->family(); + caps.sample_rate = speech_only->sample_rate(); + caps.requires_reference_audio = speech_only->requires_reference_audio(); + std::cerr << "server: speech/music-only model (family=" << caps.family << ", " + << caps.sample_rate << " Hz, " + << (caps.requires_reference_audio ? "reference clip REQUIRED" + : "text-only synthesis") + << ", family " + << (args.speech_family.empty() ? "DETECTED" : "DECLARED (--speech-family)") + << "); serving /v1/audio/speech\n"; + namespace oai = vllm::entrypoints::openai; + oai::OpenAIServingModels speech_models(speech_served_name); + oai::ApiServer speech_server(speech_models, vllm::Version()); + speech_server.set_synthesizer( + [speech_only](const vllm::openai::SpeechRequest& req) + -> vllm::openai::SpeechResponse { + return vllm::openai::SynthesizeSpeechRequest(*speech_only, req); + }, + caps); + std::cerr << "server: listening on http://" << args.host << ":" << args.port + << " (model '" << speech_served_name << "')\n"; + vllm::platform::ConsoleShutdown shutdown_on_signal( + [&]() { speech_server.stop(); }); + if (!speech_server.listen(args.host, args.port)) { + std::cerr << "server: failed to bind " << args.host << ":" << args.port << "\n"; + return 1; + } + return 0; + } + const fs::path dir = NativeUtf8Path(args.model_dir); const std::string config_path = PathUtf8(dir / "config.json"); const std::string tokenizer_path = PathUtf8(dir / "tokenizer.json"); @@ -1312,13 +1389,18 @@ int VllmServerMain(int argc, char** argv) { smp.path = args.speech_model; smp.family = args.speech_family; // empty => DETECT by inspecting the artifact std::string why; - std::unique_ptr loaded = registry.Load(smp, &why); - if (loaded == nullptr) { + // NOT `loaded`: that name is already taken by the TEXT engine in the + // enclosing scope, and MSVC's C4456 is a warning-as-error there, so the + // shadow red every pull request's `windows-msvc-*` pair with a failure + // about a line the author had not touched (#965). + std::unique_ptr loaded_speech_engine = + registry.Load(smp, &why); + if (loaded_speech_engine == nullptr) { // `why` names every family that was tried and the path, so a startup // failure is evidence rather than a verdict. throw std::runtime_error("server: --speech-model " + args.speech_model + ": " + why); } - speech_engine = std::move(loaded); + speech_engine = std::move(loaded_speech_engine); vllm::openai::SpeechCapabilities caps; caps.family = speech_engine->family(); caps.sample_rate = speech_engine->sample_rate(); diff --git a/src/vllm/entrypoints/openai/speech_api.cpp b/src/vllm/entrypoints/openai/speech_api.cpp index 4bff02d66..cece9fddc 100644 --- a/src/vllm/entrypoints/openai/speech_api.cpp +++ b/src/vllm/entrypoints/openai/speech_api.cpp @@ -193,6 +193,45 @@ SpeechRequest ParseSpeechRequest(const std::string& body) { out.has_guidance_scale = true; } if (Has(extra, "seed")) out.seed = static_cast(ReadNumber(extra, "seed")); + + // ── The keys UPSTREAM refuses, refused here too (#672) ──────────────────── + // + // SGLang-Omni's own `/v1/audio/speech` accepts these on the schema + // (`serve/protocol.py:361-364`) and then REFUSES them for MiniMax-Music3 when + // they are explicitly set — `models/minimax_music3/request_builders.py:14-19` + // lists them and `:109-114` raises + // "MiniMax Music 3 does not support sampling parameters: …". + // It refuses them because this model's AR stage has no temperature and no + // nucleus branch at all: `_sample_top_k` is the ONLY sampler either stage + // uses, and `_AR_SAMPLING_TOP_K` is a module constant of 50 + // (diffusers `encoders.py:48,94-103`). A knob that cannot be honoured must + // not return 200. + // + // We were SILENT on them, which is the #925 failure class exactly: a caller + // porting an SGLang or OpenAI recipe sends `temperature`, gets a well-formed + // WAV, and has no way to learn the knob was dropped. The cost of that + // silence is already recorded in this file, one refusal above. + for (const char* key : {"temperature", "top_p", "top_k", "repetition_penalty"}) { + VT_CHECK(!Has(extra, key), + std::string("speech request: `") + key + + "` is not supported — MiniMax-Music3's autoregressive stage has no " + "temperature and no nucleus sampling; its only sampler is a fixed top-50 " + "draw (encoders.py:48,94-103), so the knob can be neither honoured nor " + "honestly ignored. Upstream refuses it by name too " + "(request_builders.py:109-114). Use `seed` to control the draw"); + } + // SGLang-Omni spells the LENGTH as `max_new_tokens`, counted in 25 Hz audio + // FRAMES rather than in seconds (`request_builders.py:56-68`, + // `constants.py:4-5`). Our wire key is diffusers' `audio_duration`, in + // SECONDS, because diffusers is this row's primary oracle. Two duration + // spellings on one route is what #925 was, so the second one is REFUSED and + // converted for the caller rather than accepted as a near-synonym. + VT_CHECK(!Has(extra, "max_new_tokens"), + "speech request: `max_new_tokens` is SGLang-Omni's spelling of the length, counted " + "in 25 Hz audio frames (request_builders.py:56-68). This route takes " + "`audio_duration` in SECONDS instead — divide by 25 — because accepting both would " + "be two names for one meaning, and a duration key that is read by nobody is how " + "this project shipped a 750x job behind a 200 (#925)"); return out; } diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 2a0c0f819..9a00bc4b8 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -241,6 +241,12 @@ target_compile_definitions(test_minimax_music3_e2e_real PRIVATE BUILD_ARTIFACT_DIR="${CMAKE_BINARY_DIR}/music3") target_include_directories(test_minimax_music3_e2e_real PRIVATE ${CMAKE_SOURCE_DIR}/tests/parity) +# This suite binds a REAL cpp-httplib server on an ephemeral port (the +# speech-only route-table cases, #672), which is the same reason +# test_openai_api_server and test_openai_conformance carry RUN_SERIAL below: +# under a saturated `ctest -jN` the server's accept thread starves and the +# client sees a connection failure rather than the defect it was looking for. +set_tests_properties(test_minimax_music3_e2e_real PROPERTIES RUN_SERIAL ON) # ARCH-ONE-SURFACE ROW 2 fold gate: the video seam == the replicated pre-fold # pipeline == the goldens the PRE-fold minimax-h3-gen binary rendered at the # branch base (frames + WAV byte-identical, mux argv pinned). diff --git a/tests/parity/test_minimax_music3_e2e_real.cpp b/tests/parity/test_minimax_music3_e2e_real.cpp index 21c486b14..23ba132a9 100644 --- a/tests/parity/test_minimax_music3_e2e_real.cpp +++ b/tests/parity/test_minimax_music3_e2e_real.cpp @@ -55,8 +55,15 @@ #include #include #include +#include #include +// The socket cases below drive the REAL cpp-httplib transport, the same one +// `vllm-server` binds. This file already depends on the server being built +// (`ApiServer::handle_audio_speech` lives behind the same gate), so the client +// header adds no dependency the suite did not already have. +#include + #include "npy.h" #include "vllm/entrypoints/openai/api_server.h" #include "vllm/entrypoints/openai/speech_api.h" @@ -84,6 +91,26 @@ constexpr int64_t kFrameHiddenRow = 8 * 4096; constexpr int64_t kDenoiseSteps = 4; constexpr int64_t kSampleRate = 44100; +// ONE request body, used by the stub-synthesizer socket case AND by the real +// checkpoint case, so the two cannot drift into testing different requests. +// +// The duration key is `audio_duration`. This body used to spell it +// `audio_duration_s` — the name of the FIELD it fills — which the parser did not +// read, so the request fell back to the family's 60 s default: 1500 AR frames +// instead of 2, eight denoise windows instead of one, and 5167 vocoder latents +// instead of 6. That is the 750x job three runs were killed inside and reported +// as a hung weight load (#852). The parser now REFUSES the misspelling, and the +// checkpoint-free case above asserts that refusal. +constexpr const char* kMusic3RequestBody = R"({ + "model": "minimax-music3", + "lyrics": "[verse]\nMorning light\n", + "description": "Genre: acoustic pop. BPM: 96.", + "audio_duration": 0.1, + "num_inference_steps": 2, + "seed": 7, + "response_format": "wav" + })"; + // ─── The bounds ───────────────────────────────────────────────────────────── // // The DELIVERY case's bounds are W5's, unchanged and for the same reason: the @@ -323,14 +350,165 @@ uint16_t ReadU16(const std::string& wav, size_t offset) { (static_cast(wav[offset + 1]) << 8)); } +// Set by every case that actually examined the 27 GB checkpoint, and READ by +// the coverage-report case at the bottom of this file. It is what turns "this +// suite reported no failures" into "this suite reported what it examined": with +// the checkpoint present but the checkpoint cases hollowed out, the report +// REDS, and with the checkpoint absent it says so in one line a human and a log +// scraper can both find. +int g_checkpoint_arms_run = 0; + } // namespace +// --------------------------------------------------------------------------- +// THE CHECKPOINT-FREE HALF — runs in CI, unconditionally, every time. +// +// Everything below this banner needs no weights, no network and no goldens, and +// it is separated out for a reason that is not tidiness. Before it existed this +// file reported `test cases: 5 | 5 passed | 0 failed` and `assertions: 0` +// whenever the checkpoint was absent, which is a skip wearing a pass: five +// green case names, nothing examined. Splitting the suite means the parts a +// machine can always check are always checked, and the parts that need 27 GB +// are visibly the only ones that can go unexamined. +// --------------------------------------------------------------------------- + +TEST_CASE("music3 e2e: the REQUEST CONTRACT holds with no checkpoint") { + // The exact body the checkpoint case posts, parsed through the code HTTP + // runs. A field that stopped landing here would make every downstream + // assertion in this file a claim about the wrong request. + const vllm::openai::SpeechRequest request = vllm::openai::ParseSpeechRequest(R"({ + "model": "minimax-music3", + "lyrics": "[verse]\nMorning light\n", + "description": "Genre: acoustic pop. BPM: 96.", + "audio_duration": 0.1, + "num_inference_steps": 2, + "seed": 7, + "response_format": "wav" + })"); + CHECK(request.lyrics == "[verse]\nMorning light\n"); + CHECK(request.description == "Genre: acoustic pop. BPM: 96."); + CHECK(request.audio_duration_s == doctest::Approx(0.1)); + CHECK(request.num_inference_steps == 2); + CHECK(request.seed == 7); + CHECK(request.text.empty()); // Music3 REFUSES `input`; it must not arrive filled + + // The near-miss key that cost this row four multi-hour runs (#852, #925). It + // is asserted HERE, in the file whose own body carried the misspelling, so + // the regression cannot come back through the door it came through. + CHECK_THROWS(vllm::openai::ParseSpeechRequest( + R"({"lyrics":"x","audio_duration_s":0.1})")); + + // The knobs upstream refuses and we used to drop silently (#672 parity sweep). + CHECK_THROWS(vllm::openai::ParseSpeechRequest(R"({"lyrics":"x","temperature":0.7})")); + CHECK_THROWS(vllm::openai::ParseSpeechRequest(R"({"lyrics":"x","top_p":0.9})")); + CHECK_THROWS(vllm::openai::ParseSpeechRequest(R"({"lyrics":"x","max_new_tokens":250})")); + CHECK_THROWS(vllm::openai::ParseSpeechRequest(R"({"lyrics":"x","stream":true})")); + CHECK_THROWS(vllm::openai::ParseSpeechRequest(R"({"lyrics":"x","voice":"alloy"})")); + CHECK_THROWS(vllm::openai::ParseSpeechRequest(R"({"lyrics":"x","speed":1.5})")); + CHECK_THROWS( + vllm::openai::ParseSpeechRequest(R"({"lyrics":"x","response_format":"mp3"})")); +} + +TEST_CASE("music3 e2e: the DURATION arithmetic holds with no checkpoint") { + // The request's `audio_duration` becomes AR frames, and this is the step that + // turned 0.1 s into 60 s when the key was dropped. It is pure arithmetic over + // the 25 Hz frame rate, so it needs no weights — and it pins the two ceilings + // spec §7 says are "enforced, not discovered". + constexpr double kFrameRate = 25.0; // condition_encoder 24000 / 960 + CHECK(m3::MaxArFrames(0.1, kFrameRate) == 2); + CHECK(m3::MaxArFrames(60.0, kFrameRate) == 1500); // the family default + // The 9000-frame ceiling CLAMPS rather than throwing, which is upstream's own + // behaviour (`min(int(audio_duration * frame_rate), 9000)`, encoders.py:287). + // Refusing where upstream clamps would be a divergence a user hits at six + // minutes of music. + CHECK(m3::MaxArFrames(360.0, kFrameRate) == 9000); + CHECK(m3::MaxArFrames(3600.0, kFrameRate) == 9000); + // Below one frame, and non-positive, are upstream's two errors. + CHECK_THROWS(m3::MaxArFrames(0.01, kFrameRate)); + CHECK_THROWS(m3::MaxArFrames(0.0, kFrameRate)); + CHECK_THROWS(m3::MaxArFrames(-1.0, kFrameRate)); +} + +TEST_CASE("music3 e2e: a SPEECH-ONLY server serves a WAV over a real socket") { + // `vllm-server --speech-model ` with NO `--model` (#672). The ROUTE + // TABLE that invocation produces, over a real socket, with a stub synthesizer + // so it needs no weights: the music route is ours and the text routes are + // ABSENT rather than present-and-broken. + // + // The construction is the server's own: an `ApiServer` from serving-models + // alone — no completion handler, no chat handler — plus a synthesizer. The + // checkpoint case below builds the identical shape around the REAL engine, so + // this one gates the shape and that one gates the weights. + vllm::entrypoints::openai::OpenAIServingModels models{"minimax-music3"}; + vllm::entrypoints::openai::ApiServer server{models, "music3-speech-only"}; + vllm::openai::SpeechCapabilities caps; + caps.family = "minimax-music3"; + caps.sample_rate = kSampleRate; + caps.channels = kWaveformChannels; + caps.requires_reference_audio = false; + server.set_synthesizer( + [](const vllm::openai::SpeechRequest&) { + vllm::openai::SpeechResponse out; + std::vector samples(2 * 1024, 0.25F); + out.wav = ::vllm::MiniMaxH3WriteWav(samples, kWaveformChannels, 1024, kSampleRate); + out.sample_rate = kSampleRate; + out.channels = kWaveformChannels; + out.samples_per_channel = 1024; + return out; + }, + caps); + + const int port = server.bind_to_any_port("127.0.0.1"); + REQUIRE(port > 0); + std::thread server_thread([&server]() { server.serve(); }); + for (int i = 0; i < 500 && !server.is_running(); ++i) { + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + } + REQUIRE(server.is_running()); + { + httplib::Client client("127.0.0.1", port); + client.set_connection_timeout(5, 0); + client.set_read_timeout(30, 0); + auto speech = client.Post("/v1/audio/speech", kMusic3RequestBody, "application/json"); + REQUIRE(speech); + CHECK(speech->status == 200); + CHECK(speech->get_header_value("Content-Type") == "audio/wav"); + REQUIRE(speech->body.size() > 44); + CHECK(speech->body.compare(0, 4, "RIFF") == 0); + CHECK(ReadU16(speech->body, 22) == kWaveformChannels); + CHECK(ReadU32(speech->body, 24) == kSampleRate); + + // A well-formed generate body — exactly what the route would accept if it + // existed — falls through to httplib's own 404. That is what proves the + // route was never REGISTERED, rather than that a handler rejected it. + auto completions = client.Post("/v1/completions", + R"({"model":"minimax-music3","prompt":"hi"})", + "application/json"); + REQUIRE(completions); + CHECK(completions->status == 404); + auto chat = client.Post( + "/v1/chat/completions", + R"({"model":"minimax-music3","messages":[{"role":"user","content":"hi"}]})", + "application/json"); + REQUIRE(chat); + CHECK(chat->status == 404); + + // And the server is alive, so those 404s are about the routes. + auto health = client.Get("/health"); + REQUIRE(health); + CHECK(health->status == 200); + } + server.stop(); + server_thread.join(); +} + // --------------------------------------------------------------------------- // The DELIVERY path: latents -> waveform -> the WAV a request receives // --------------------------------------------------------------------------- TEST_CASE("music3 e2e real: the decode reaches waveform.npy at 44100 stereo") { if (SkipIfMissing("music3 e2e decode")) return; + ++g_checkpoint_arms_run; const vllm::MiniMaxMusic3Config config = vllm::MiniMaxMusic3LoadConfig(Paths()); // spec §1.1, read from the CHECKPOINT rather than assumed by this file. CHECK(config.vocoder.sampling_rate == kSampleRate); @@ -377,6 +555,7 @@ TEST_CASE("music3 e2e real: the decode reaches waveform.npy at 44100 stereo") { TEST_CASE("music3 e2e real: the WAV a request receives is 44100 Hz 16-bit STEREO") { if (SkipIfMissing("music3 e2e wav")) return; + ++g_checkpoint_arms_run; std::vector shape; const std::vector waveform = LoadF32Npy("waveform.npy", &shape); REQUIRE(shape[0] == kWaveformChannels); @@ -427,6 +606,7 @@ TEST_CASE("music3 e2e real: the WAV a request receives is 44100 Hz 16-bit STEREO TEST_CASE("music3 e2e real: the tail's first stage still reproduces condition_chunk0") { if (SkipIfMissing("music3 e2e condition")) return; + ++g_checkpoint_arms_run; std::vector shape; const std::vector frame_hiddens = LoadF32Npy("frame_hiddens.npy", &shape); REQUIRE(shape.size() == 2); @@ -457,6 +637,7 @@ TEST_CASE("music3 e2e real: the tail's first stage still reproduces condition_ch TEST_CASE("music3 e2e real: the whole tail reaches waveform.npy from frame_hiddens") { if (SkipIfMissing("music3 e2e full tail")) return; if (SkipIfDitNotRequested("music3 e2e full tail")) return; + ++g_checkpoint_arms_run; const vllm::MiniMaxMusic3Config config = vllm::MiniMaxMusic3LoadConfig(Paths()); std::vector shape; @@ -563,18 +744,26 @@ TEST_CASE("music3 e2e real: the whole tail reaches waveform.npy from frame_hidde // splits past 200 frames, which is 8 s of audio and hours of scalar CPU, and // the multi-window composition is W6's named coverage gap either way. // -// ─── WHY IT GOES THROUGH `handle_audio_speech` ────────────────────────────── +// ─── WHY IT GOES OVER A REAL SOCKET ───────────────────────────────────────── +// +// Because the claim is about an HTTP request, and the shape a user runs is +// `vllm-server --speech-model ` with no `--model` at all (#672). This case +// builds exactly what that invocation builds — an `ApiServer` from +// serving-models alone, no completion handler and no chat handler, plus the +// REAL loaded engine behind `vllm::openai::SynthesizeSpeechRequest`, which is +// the same function `server_main.cpp` hands it — binds an ephemeral port and +// posts over the wire. // -// Because the claim is about an HTTP request. `ApiServer::handle_audio_speech` -// is the route's own body: it parses the JSON, applies the -// `requires_reference_audio` pre-refusal, calls the synthesizer and sets the -// `audio/wav` content type. The synthesizer it is handed here is -// `vllm::openai::SynthesizeSpeechRequest` — the SAME function `server_main.cpp` -// hands it — so nothing on this path is a test-only reimplementation of the -// server. -TEST_CASE("music3 e2e real: an HTTP request generates a real 44100 Hz stereo WAV") { +// A direct `handle_audio_speech` call, which is what this case used to do, +// exercises the handler but never the ROUTE TABLE: it cannot tell a registered +// route from an unregistered one, and the speech-only server's whole claim is +// about which routes exist. The checkpoint-free twin at the top of this file +// gates that table with a stub; this one gates it with 28.5 GB of weights +// behind it. +TEST_CASE("music3 e2e real: a music-only server generates a real 44100 Hz stereo WAV") { if (SkipIfMissing("music3 e2e http")) return; if (SkipIfDitNotRequested("music3 e2e http")) return; + ++g_checkpoint_arms_run; vllm::multimodal::SpeechRegistry registry; m3::RegisterBuiltinSpeechFamilies(registry); @@ -601,35 +790,73 @@ TEST_CASE("music3 e2e real: an HTTP request generates a real 44100 Hz stereo WAV }, caps); - // The body a client posts. `lyrics` and `description` are the two music - // inputs; `input` is deliberately absent, because Music3 REFUSES it. + // ── over the wire, on an ephemeral port ────────────────────────────────── // - // The duration key is `audio_duration`, which is what `ParseSpeechRequest` - // reads and what docs/USAGE.md documents. This body used to spell it - // `audio_duration_s` — the name of the FIELD it fills — which the parser did - // not read, so the request fell back to the family's 60 s default: 1500 AR - // frames instead of 2, eight denoise windows instead of one, and 5167 vocoder - // latents instead of 6. That is the 750x job three runs were killed inside - // and reported as a hung weight load (#852). The parser now REFUSES the - // misspelling instead of defaulting past it. - const std::string body = R"({ - "model": "minimax-music3", - "lyrics": "[verse]\nMorning light\n", - "description": "Genre: acoustic pop. BPM: 96.", - "audio_duration": 0.1, - "num_inference_steps": 2, - "seed": 7, - "response_format": "wav" - })"; + // The body is `kMusic3RequestBody`, shared with the stub-synthesizer case at + // the top of this file so the two cannot drift into testing different + // requests. `lyrics` and `description` are the two music inputs; `input` is + // deliberately absent, because Music3 REFUSES it. + // + // The read timeout is generous because this synthesis is 8.6B + 2.4B of + // scalar host float on CPU: a 0.1 s request is minutes, not milliseconds, and + // a client timeout would present as a transport failure rather than as the + // slow model it is. + const int port = server.bind_to_any_port("127.0.0.1"); + REQUIRE(port > 0); + std::thread server_thread([&server]() { server.serve(); }); + for (int i = 0; i < 500 && !server.is_running(); ++i) { + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + } + REQUIRE(server.is_running()); + + std::string wav; + int status = 0; + std::string content_type; + int completions_status = 0; + int chat_status = 0; + { + httplib::Client client("127.0.0.1", port); + client.set_connection_timeout(10, 0); + client.set_read_timeout(7200, 0); // hours: this is scalar fp32 on CPU + + // The TEXT routes first, and BEFORE the long synthesis, so the route-table + // claim is recorded even if the generation is interrupted. + auto completions = client.Post("/v1/completions", + R"({"model":"minimax-music3","prompt":"hi"})", + "application/json"); + REQUIRE(completions); + completions_status = completions->status; + auto chat = client.Post( + "/v1/chat/completions", + R"({"model":"minimax-music3","messages":[{"role":"user","content":"hi"}]})", + "application/json"); + REQUIRE(chat); + chat_status = chat->status; + + const auto t0 = std::chrono::steady_clock::now(); + auto response = client.Post("/v1/audio/speech", kMusic3RequestBody, "application/json"); + const double seconds = + std::chrono::duration(std::chrono::steady_clock::now() - t0).count(); + REQUIRE_MESSAGE(response, "POST /v1/audio/speech did not complete over the socket"); + status = response->status; + content_type = response->get_header_value("Content-Type"); + wav = response->body; + MESSAGE("POST /v1/audio/speech -> " << status << " " << content_type << ", " << wav.size() + << " bytes in " << seconds << " s wall"); + } + server.stop(); + server_thread.join(); - const vllm::entrypoints::openai::ApiServer::DispatchResult response = server.handle_audio_speech(body); - MESSAGE("POST /v1/audio/speech -> " << response.status << " " << response.content_type << ", " - << response.body.size() << " bytes"); - REQUIRE_MESSAGE(response.status == 200, response.body); - CHECK(response.content_type == "audio/wav"); + // A music-only server registers ONE route and NOT the generate pair. A + // well-formed generate body falling through to httplib's own 404 is what + // proves the route was never registered. + CHECK(completions_status == 404); + CHECK(chat_status == 404); + + REQUIRE_MESSAGE(status == 200, wav); + CHECK(content_type == "audio/wav"); // ── the RIFF header ────────────────────────────────────────────────────── - const std::string& wav = response.body; REQUIRE(wav.size() > 44); CHECK(wav.compare(0, 4, "RIFF") == 0); CHECK(wav.compare(8, 4, "WAVE") == 0); @@ -714,3 +941,92 @@ TEST_CASE("music3 e2e real: an HTTP request generates a real 44100 Hz stereo WAV MESSAGE("wrote " << wav.size() << " bytes to " << out.string()); CHECK(fs::is_regular_file(out, ec)); } + +// --------------------------------------------------------------------------- +// THE COVERAGE REPORT — registered LAST, and the reason it exists +// +// With neither `VLLM_CPP_MUSIC3_CHECKPOINT` nor `VLLM_CPP_MUSIC3_DIT` set, this +// file used to report: +// +// test cases: 5 | 5 passed | 0 failed | 0 skipped +// assertions: 0 | 0 passed | 0 failed +// +// Five green case names over an empty run. That is a SKIP WEARING A PASS, and +// this project has already been fooled by the identical shape on +// test_qwen3_paged_engine, which "passes 2/2" while asserting nothing because +// its snapshots are dgx-only. A gate that cannot say HOW MANY things it +// examined has not reported. +// +// Two changes fix it, and both are needed: +// +// 1. the checkpoint-free half at the top of this file runs unconditionally, +// so `assertions: 0` is now structurally impossible — the suite always +// examines the request contract, the duration arithmetic and the +// speech-only route table, and REDS in CI if any of them breaks; +// 2. this case prints WHICH arms ran, every run, and asserts the one +// invariant a hollowed-out suite would violate: with the checkpoint +// PRESENT, at least one checkpoint arm must have executed. Delete the +// bodies of the four cases above and this one goes RED on a box that has +// the weights, where a case count alone would still read green. +// +// It cannot fail on a box that legitimately lacks the 27 GB checkpoint, because +// that is not a defect — it is the reason the env gate exists. What it refuses +// to do is stay silent about it. +// --------------------------------------------------------------------------- +TEST_CASE("music3 e2e: THE COVERAGE THIS RUN ACTUALLY TOOK, reported not implied") { + const std::string reason = MissingReason(); + const char* dit = std::getenv("VLLM_CPP_MUSIC3_DIT"); + const bool dit_on = dit != nullptr && std::string(dit) != "0" && !std::string(dit).empty(); + + std::printf( + "[music3 e2e COVERAGE] checkpoint=%s dit=%s checkpoint_arms_run=%d\n", + reason.empty() ? "PRESENT" : "ABSENT", dit_on ? "ON" : "OFF", g_checkpoint_arms_run); + if (!reason.empty()) { + std::printf( + "[music3 e2e COVERAGE] the CHECKPOINT arms did not run: %s\n" + "[music3 e2e COVERAGE] what DID run is the checkpoint-free half — the request\n" + "[music3 e2e COVERAGE] contract, the duration arithmetic and the speech-only route\n" + "[music3 e2e COVERAGE] table over a real socket. Set VLLM_CPP_MUSIC3_CHECKPOINT (and\n" + "[music3 e2e COVERAGE] VLLM_CPP_MUSIC3_DIT=1) to run the rest.\n", + reason.c_str()); + } else if (!dit_on) { + std::printf( + "[music3 e2e COVERAGE] the two DiT arms did not run: VLLM_CPP_MUSIC3_DIT is unset\n" + "[music3 e2e COVERAGE] (eight 2.4B fp32 host forwards, hours on CPU).\n"); + } + const std::string checkpoint_state = reason.empty() ? std::string("PRESENT") : reason; + const std::string dit_state = dit_on ? std::string("ON") : std::string("OFF"); + MESSAGE("coverage: checkpoint=" << checkpoint_state << " dit=" << dit_state + << " checkpoint_arms_run=" << g_checkpoint_arms_run); + + // ── The invariant, and why it asserts the CHECKPOINT rather than the counter + // + // The obvious guard is `g_checkpoint_arms_run > 0` when the checkpoint is + // present. It is REJECTED here, because it is a claim about test ORDERING + // rather than about the tree: `-tc="…COVERAGE…"` runs this case alone, the + // counter is legitimately 0, and the gate reds for the filter rather than for + // a defect. A gate that reds for the way it was invoked is a gate someone + // deletes. + // + // What is asserted instead is a fact about the checkpoint ITSELF, read from + // the seven component `config.json` files — milliseconds, no weight byte — + // so it holds under any invocation and reds only if the checkpoint the other + // cases were about is absent, unreadable or not this model. The counter stays + // as REPORTED context above, which is what it is good for. + if (reason.empty()) { + const vllm::MiniMaxMusic3Config config = vllm::MiniMaxMusic3LoadConfig(Paths()); + // 44100 stereo is spec §1.1's resolved rate, and it is DERIVED rather than + // declared: 8*8*4*2 = 512 upsampling over 86.133 Hz latent frames. + CHECK(config.vocoder.sampling_rate == kSampleRate); + CHECK(config.vocoder.hop_length() == 512); + // The 200000-entry music vocabulary is what makes this Music3's language + // model rather than a stock Qwen3. + CHECK(config.language_model.vocab_size == 200000); + // And the eight RVQ codebooks the depth decoder expands each frame into. + CHECK(config.rvq_depth_decoder.num_codebooks == 8); + } else { + // Still an assertion, so this case can never contribute zero: an absence + // reported with no reason is the thing this whole case exists to prevent. + CHECK(reason.find_first_not_of(" \t") != std::string::npos); + } +} diff --git a/tests/vllm/entrypoints/openai/test_api_server.cpp b/tests/vllm/entrypoints/openai/test_api_server.cpp index fe04249ae..93a3f75bf 100644 --- a/tests/vllm/entrypoints/openai/test_api_server.cpp +++ b/tests/vllm/entrypoints/openai/test_api_server.cpp @@ -3125,3 +3125,99 @@ TEST_CASE("api_server: /v1/audio/speech route registration is ADDITIVE over a re }); } } + +// ─── The SPEECH-ONLY server (#672) ────────────────────────────────────────── +// +// `vllm-server --speech-model ` with NO `--model` is now a valid +// invocation, because upstream's own recipe is `sgl-omni serve --model +// MiniMaxAI/MiniMax-Music3` and nothing else — a music model is not an +// accessory to a text model. Requiring a text checkpoint beside a 28.5 GB music +// one made the documented recipe unrunnable on any box whose smallest text +// model is tens of gigabytes. +// +// This case pins the ROUTE TABLE that invocation produces, which is the half a +// handler-dispatch test cannot see. It is the exact twin of the ASR pair above +// ("transcriptions socket smoke … generate routes 404" / "the audio routes do +// not exist on a TEXT server"), and it uses the SAME construction the server's +// speech-only branch does: an `ApiServer` built from serving-models alone, with +// no completion and no chat handler, plus a synthesizer. +// +// It needs no checkpoint and no engine, so it runs in CI unconditionally. +TEST_CASE("api_server: a SPEECH-ONLY server serves speech and 404s the generate routes") { + OpenAIServingModels models{"minimax-music3"}; + ApiServer server{models, "speech-only"}; + vllm::openai::SpeechCapabilities caps; + caps.family = "minimax-music3"; + caps.sample_rate = 44100; + caps.channels = 2; + caps.requires_reference_audio = false; + server.set_synthesizer([](const vllm::openai::SpeechRequest&) { return StereoWav(1024); }, + caps); + + const int port = server.bind_to_any_port("127.0.0.1"); + REQUIRE(port > 0); + std::thread server_thread([&server]() { server.serve(); }); + for (int i = 0; i < 500 && !server.is_running(); ++i) + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + REQUIRE(server.is_running()); + + { + httplib::Client client("127.0.0.1", port); + client.set_connection_timeout(5, 0); + client.set_read_timeout(15, 0); + + // The route this server exists to serve. + auto speech = client.Post("/v1/audio/speech", MusicBody(), "application/json"); + REQUIRE(speech); + CHECK(speech->status == 200); + CHECK(speech->get_header_value("Content-Type") == "audio/wav"); + REQUIRE(speech->body.size() == 44u + 1024u * 2u * 2u); + CHECK(speech->body.compare(0, 4, "RIFF") == 0); + CHECK(WavU16(speech->body, 22) == 2); // STEREO + CHECK(WavU32(speech->body, 24) == 44100u); // the family's NATIVE rate + + // The TEXT routes are ABSENT, not present-and-broken. A well-formed body — + // exactly what the route would accept if it existed — must fall through to + // httplib's own 404, which is what proves the route was never registered + // rather than that a handler rejected the payload. + auto completions = client.Post("/v1/completions", + R"({"model":"minimax-music3","prompt":"hi"})", + "application/json"); + REQUIRE(completions); + CHECK(completions->status == 404); + auto chat = client.Post( + "/v1/chat/completions", + R"({"model":"minimax-music3","messages":[{"role":"user","content":"hi"}]})", + "application/json"); + REQUIRE(chat); + CHECK(chat->status == 404); + // Not OUR 404 either: no error envelope is emitted, so nothing tells a + // client the route half-exists. + CHECK(completions->body.find("\"object\"") == std::string::npos); + CHECK(chat->body.find("\"object\"") == std::string::npos); + + // The OTHER opt-in surfaces stay unregistered too — attaching a synthesizer + // adds ONE endpoint, not a family of them. + auto videos = client.Post("/v1/videos", R"({"prompt":"x"})", "application/json"); + REQUIRE(videos); + CHECK(videos->status == 404); + auto embeddings = + client.Post("/v1/embeddings", R"({"input":"x"})", "application/json"); + REQUIRE(embeddings); + CHECK(embeddings->status == 404); + + // Liveness and discovery still serve, so the 404s above are about the + // routes and not about a dead server — and `/v1/models` reports the FAMILY, + // which is what the speech-only branch defaults the served name to. + auto health = client.Get("/health"); + REQUIRE(health); + CHECK(health->status == 200); + auto models_res = client.Get("/v1/models"); + REQUIRE(models_res); + CHECK(models_res->status == 200); + CHECK(json::parse(models_res->body).at("data").at(0).at("id") == "minimax-music3"); + } + + server.stop(); + server_thread.join(); +} diff --git a/tests/vllm/entrypoints/openai/test_speech_api.cpp b/tests/vllm/entrypoints/openai/test_speech_api.cpp index 36d3e9030..afe2fc9f3 100644 --- a/tests/vllm/entrypoints/openai/test_speech_api.cpp +++ b/tests/vllm/entrypoints/openai/test_speech_api.cpp @@ -169,6 +169,50 @@ TEST_CASE("speech api: every unsupported field is REFUSED, never ignored") { CHECK_NOTHROW(ParseSpeechRequest(R"({"input":"hi","response_format":"wav"})")); } +// The parity sweep of #672: SGLang-Omni serves this same model on this same +// route and REFUSES four sampling knobs and its own length spelling. We were +// SILENT on all five, which is the #925 failure class one case above — a knob +// the server will not honour must not come back behind a 200. +TEST_CASE("speech api: the sampling knobs UPSTREAM refuses are refused here too") { + // `request_builders.py:14-19,109-114`. This model's AR stage has ONE sampler, + // a fixed top-50 draw (`encoders.py:48,94-103`): there is no temperature to + // set and no nucleus branch to widen, so honouring any of these is impossible + // and ignoring them is dishonest. + CHECK_THROWS_WITH(ParseSpeechRequest(R"({"lyrics":"x","temperature":0.7})"), + doctest::Contains("`temperature` is not supported")); + CHECK_THROWS_WITH(ParseSpeechRequest(R"({"lyrics":"x","top_p":0.9})"), + doctest::Contains("`top_p` is not supported")); + CHECK_THROWS_WITH(ParseSpeechRequest(R"({"lyrics":"x","top_k":40})"), + doctest::Contains("`top_k` is not supported")); + CHECK_THROWS_WITH(ParseSpeechRequest(R"({"lyrics":"x","repetition_penalty":1.1})"), + doctest::Contains("`repetition_penalty` is not supported")); + // Nested under `extra_params` too, because that is the second place every + // other generation knob on this route is read from. + CHECK_THROWS_WITH(ParseSpeechRequest(R"({"lyrics":"x","extra_params":{"temperature":0.7}})"), + doctest::Contains("`temperature` is not supported")); + + // `max_new_tokens` is SGLang-Omni's LENGTH, in 25 Hz audio frames rather than + // in seconds (`request_builders.py:56-68`). Accepting it as a near-synonym of + // `audio_duration` would put two spellings of one meaning on one route, and + // dropping it silently is how a 250-frame (10 s) request becomes the family's + // 60 s default. The refusal names the key to use AND the conversion. + CHECK_THROWS_WITH(ParseSpeechRequest(R"({"lyrics":"x","max_new_tokens":250})"), + doctest::Contains("`max_new_tokens` is SGLang-Omni's spelling")); + CHECK_THROWS_WITH(ParseSpeechRequest(R"({"lyrics":"x","max_new_tokens":250})"), + doctest::Contains("`audio_duration` in SECONDS")); + + // The NEGATIVE side, because a guard that fired on an ordinary request would + // refuse every real one: a body carrying none of the five still parses, and + // the knobs this route DOES honour are untouched by the guards above. + const SpeechRequest ok = ParseSpeechRequest( + R"({"lyrics":"[Verse]\nx\n","description":"pop","audio_duration":12.5, + "num_inference_steps":8,"guidance_scale":1.7,"seed":7})"); + CHECK(ok.audio_duration_s == doctest::Approx(12.5)); + CHECK(ok.num_inference_steps == 8); + CHECK(ok.guidance_scale == doctest::Approx(1.7)); + CHECK(ok.seed == 7); +} + TEST_CASE("speech api: a malformed body is a 400, never a silent default") { CHECK_THROWS(ParseSpeechRequest("not json")); CHECK_THROWS(ParseSpeechRequest("[]"));