spec(MODEL-MUSIC-MUSIC3): Music3 is a SpeechEngine family, not a new engine (#672) - #700
Merged
Conversation
…engine (#672) `multimodal::SpeechEngine` did not exist when this spec was written. The IndexTTS-2.5 lane landed it on 2026-08-13, along with `Vocoder1D`, while W0 was in review. AGENTS.md is explicit that a capability not reachable through the shared surface is not done, and that a seam is EXTENDED rather than forked -- so the scope changes before an implementer builds the wrong thing, not after. IT FITS BETTER THAN IT LOOKS. `SpeechResult` already carries `channels`, and already documents `sample_rate` as "the family's native rate ... rather than a resampled one, so the caller decides whether to resample". That is exactly §1.1's finding arrived at independently: 44100 stereo is the model's rate, and SGLang-Omni's 32 kHz is a caller-side transform. `requires_reference_audio()` exists so a server can refuse before staging weights; Music3 returns false where IndexTTS-2 returns true, which is the difference between a voice cloner and a text-to-music model expressed in the seam rather than around it. THE ONE GENUINE GAP IS `SpeechGenParams`, which carries a single `text` field because IndexTTS-2 synthesises one utterance. Music3 takes TWO distinct inputs -- lyrics with section tags, and a structured music description -- plus generation controls (duration/frames, denoise steps, CFG). Packing both into `text` behind a separator would be a private protocol inside a shared struct, which is precisely the fork the rule exists to prevent. W6 extends the struct ADDITIVELY and leaves IndexTTS-2.5 byte-identical: a field an existing family ignores costs it nothing, while a second parallel params struct costs every future family a choice. If the extension cannot be made additive, that is a NEEDS_DECISION, not a fork. ALSO RECORDED: `SpeechEngine` is not on the ABI yet. include/vllm.h is at v18 and exposes the video engine but no `vllm_speech_*` surface, and no open PR adds one (checked the 15 open PRs). So W6 owns that surface, its version bump, and the example HTTP server as a thin client of it -- and the W5 row now names `Vocoder1D` as the thing to route through rather than leaving "the vocoder" to be invented. W5 and W6 both gain the same obligation as a done-when: H3 and IndexTTS-2.5 behaviour stays byte-identical. Touching a seam another lane owns is allowed; moving its numerics silently is not. Issue: #672 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec-only. Retargets Music3's W5/W6 onto the
SpeechEngineseam that theIndexTTS-2.5 lane landed on main after #679 was written, so an implementer
builds the right thing rather than a parallel path we unwind at W6.
Why now
multimodal::SpeechEngine+SpeechRegistry+Vocoder1Dlanded 2026-08-13while W0 was in review. AGENTS.md: a capability not reachable through the shared
surface is not done, and a seam is extended rather than forked.
It fits better than it looks
SpeechResultalready carrieschannels, and already documentssample_rateas"the family's native rate … rather than a resampled one, so the caller decides
whether to resample." That is #679's §1.1 finding arrived at independently:
44100 stereo is the model's rate and SGLang-Omni's 32 kHz is a caller-side
transform.
requires_reference_audio()returnsfalsefor Music3 andtrueforIndexTTS-2 — the difference between a voice cloner and a text-to-music model,
expressed in the seam instead of around it.
The one real gap
SpeechGenParamscarries a singletextfield. Music3 takes two distinctinputs — lyrics with
[Verse]/[Chorus]tags, and a structured musicdescription — plus generation controls. Packing both into
textbehind aseparator would be a private protocol inside a shared struct.
W6 extends it additively, leaving IndexTTS-2.5 byte-identical. A field an
existing family ignores costs it nothing; a second params struct costs every
future family a choice. If it cannot be additive, that is
NEEDS_DECISION, not afork.
Also recorded
SpeechEngineis not on the ABI:include/vllm.his at v18 with a videoengine and no
vllm_speech_*surface, and none of the 15 open PRs adds one. W6owns that surface, the version bump, and the example HTTP server as a thin client.
W5 and W6 gain a shared done-when: H3 and IndexTTS-2.5 behaviour stays
byte-identical. Touching another lane's seam is allowed; moving its numerics
silently is not.
Gate:
check-agent-record.pyOK (MODEL=373),check-doc-checkpoint.pyOK. Nosrc/,include/ortests/change.🤖 Generated with Claude Code