Goal
Route Gemma-4 FP8 MoE T=2..63 through the existing per-token indexed helpers so short warmups skip today's serial RunGemma4Fp8TopKOnExpertDevice path. Port lab VT_GEMMA4_DECODE_INDEXED_MAX_T default 63.
Hypothesis (A) — cause unconfirmed
T=19 warmup is observed on the serial M1 route. That does not prove the serial path is the hang cause. Distinct from the T≥64 prefill-helper accumulation class. Do not call the serial path "racy."
Context
origin/main gemma4_moe.cpp:735 takes indexed only at T == 1. T=2..63 falls through to the serial loop (:1345) RunGemma4Fp8TopKOnExpertDevice. Donor bytes are pinned under .agents/evidence/rocm-gemma4-indexed-max-t/ (dirty lab 2bb4bd8a plus uncommitted; HEAD is not a clean donor). Indexed helpers already exist on main (rocm_gemma4_experts.hip:543).
Separate from #697, #837, #839. Independent branch row/ROCM-GEMMA4-INDEXED-MAX-T.
Direction: research 713f / 5071 / 64cb + coord 25c9.
Scope
- Widen the existing T==1 indexed gate to
1 <= T <= indexed_max_t && T < kPrefillBatchMinT (64).
- Env
VT_GEMMA4_DECODE_INDEXED_MAX_T: unset → 63; =1 → T=1 only; clamp [1,63].
- T=1 keeps hipGraph-stable TLS acc; T>1 uses an owned
[T,H] buffer.
- Per-token existing indexed helpers only (same-dev and peer).
- Direct tensor oracle T=2,19,63 × {same-dev, peer} vs today's serial/reference math.
Out of scope
Gates
- Host predicate: T=1 indexed; T=19 indexed at default 63; T=64 not indexed (prefill-batch); env=1 ⇒ T=19 not indexed.
- Tensor oracle GREEN before claiming A GREEN (Paris/arith is not enough).
- T=19 generate succeeds; T=1 decode / Paris / arith unchanged.
- Do not claim p42k from this row.
Kind: bug
Goal
Route Gemma-4 FP8 MoE T=2..63 through the existing per-token indexed helpers so short warmups skip today's serial
RunGemma4Fp8TopKOnExpertDevicepath. Port labVT_GEMMA4_DECODE_INDEXED_MAX_Tdefault 63.Hypothesis (A) — cause unconfirmed
T=19 warmup is observed on the serial M1 route. That does not prove the serial path is the hang cause. Distinct from the T≥64 prefill-helper accumulation class. Do not call the serial path "racy."
Context
origin/maingemma4_moe.cpp:735takes indexed only atT == 1. T=2..63 falls through to the serial loop (:1345)RunGemma4Fp8TopKOnExpertDevice. Donor bytes are pinned under.agents/evidence/rocm-gemma4-indexed-max-t/(dirty lab2bb4bd8aplus uncommitted; HEAD is not a clean donor). Indexed helpers already exist on main (rocm_gemma4_experts.hip:543).Separate from #697, #837, #839. Independent branch
row/ROCM-GEMMA4-INDEXED-MAX-T.Direction: research
713f/5071/64cb+ coord25c9.Scope
1 <= T <= indexed_max_t && T < kPrefillBatchMinT(64).VT_GEMMA4_DECODE_INDEXED_MAX_T: unset → 63;=1→ T=1 only; clamp [1,63].[T,H]buffer.Out of scope
ExpertGeGLUFp8TopKIndexedBatched/ packed indexed-batch (already REJECTED).VT_GEMMA4_PREFILL_DEVICE_GROUP,VT_GEMMA4_PREFILL_INDEXED_NOSYNC.Gates
Kind: bug