Skip to content

The Muse Glimmer perception encoder has no production caller: an image or video prompt reaches a brick, and every change inside the tower lands unreached #1566

Description

@localai-bot

MuseGlimmerVisionForward — the whole Muse Glimmer perception encoder, its
patchify, its positional interpolation, its 2D RoPE, its 50 attention blocks and
its pixel shuffle — has no caller in src/, in examples/ or in
include/vllm.h
. The only callers in the tree are tests.

What is unreached, exactly

Read at 04f1cead6:

  • MuseGlimmerEncodePixelGroups (src/vllm/model_executor/models/muse_glimmer_mm.cpp:191)
    and MuseGlimmerGenerateGreedyViaRegistry (:276) are declared in
    include/vllm/model_executor/models/muse_glimmer.h:405,427 and called only
    from tests/vllm/models/test_muse_glimmer_wiring.cpp:724,804,805,864,883.
  • MuseGlimmerVisionForward (include/vllm/model_executor/models/muse_glimmer_vision.h:186)
    is called from muse_glimmer_mm.cpp:203 and from
    tests/vllm/models/test_muse_glimmer_vision.cpp:308,348, and nowhere else.
  • The registry TU says so in prose at
    src/vllm/model_executor/models/muse_glimmer_registry.cpp:13-14: "The
    perception encoder is still W3, so an image or video prompt is a pending
    brick."

So an image or video prompt arriving through ModelRegistry::Forward does not
reach any of it. docs/FEATURES.md:150 already records the state from the
correctness side ("vision: no reference run of any kind"); this issue records
it from the reachability side, which is a different gap with a different fix.

Why it needs an open issue of its own

AGENTS.md "Nothing lands dead" permits a staged slice to land unreached only
when the commit body, the pull request body and the row's spec ## Owed name
what is unreached, the row that owns the wiring, and the issue that tracks
it
. The model's umbrella issue
#268 is CLOSED, so until now
there was nothing open to name — and every change landing inside this slice
needs one.

The owning row is
MODEL-MM-muse-glimmer-muse-glimmer-for-conditional-generation; the work is W4
and W5 in .agents/specs/muse-glimmer.md §3.

What closing it requires

An image prompt through a production entry point — the loader, the server path,
or ModelRegistry::Forward on its default configuration — reaching
MuseGlimmerVisionForward, with placeholder expansion and the scatter into the
text sequence, and a test that enters through that entry point rather than
constructing the tower by hand. §10.4 of the spec records the separate
mmproj refusal that blocks the GGUF arm of the same capability; the bf16
safetensors arm is the one this issue is about.

Filed while landing #1545,
which changes one line inside this unreached slice and needs an issue to
name for its staged-slice exception.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions