Backend sampling multi output

[gaugarg-nv]

Overview

This PR extends backend sampling from one output per sequence to multiple outputs, enabling backend sampling during speculative decoding.

• Add n_sampling_outputs_per_seq_max to context parameters, separating the total output budget from the per-sequence backend-sampling limit.
• Build one backend-sampling graph row per output rather than per sequence.
• Extend graph reservation and node budgeting for replicated sampler graphs.
• Add graph probing for both backend operation support and sampler node counting.
• Add a backend_reset hook to clear graph-owned tensor references before graph reconstruction.
• Make distribution sampling deterministic between CPU and backend paths
• Clamp the distribution mask sum before converting it into a sampled index, preventing an out-of-range access. (Copied from sampling : support multiple outputs per sequence #19833)
• Keep output-view sources alive until graph completion, preventing allocator reuse from corrupting sampling outputs.

Additional information

Performance: I see a perf improvement of ~8% on RTX 5090 with the Qwen-3.6-35B Q4_K_M model. Acceptance ratio with both backend and CPU sampling is exactly same.

CPU sampling: llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_M.gguf --spec-type draft-mtp --seed 42

python3 mtp-bench.py
  code_python        pred= 192 draft= 168 acc= 135 rate=0.804 tok/s=379.5
  code_cpp           pred= 192 draft= 190 acc= 127 rate=0.668 tok/s=343.1
  explain_concept    pred= 192 draft= 210 acc= 119 rate=0.567 tok/s=308.9
  summarize          pred= 192 draft= 169 acc= 134 rate=0.793 tok/s=380.3
  qa_factual         pred= 192 draft= 170 acc= 134 rate=0.788 tok/s=382.0
  translation        pred= 192 draft= 176 acc= 132 rate=0.750 tok/s=370.2
  creative_short     pred= 192 draft= 207 acc= 120 rate=0.580 tok/s=315.8
  stepwise_math      pred= 192 draft= 160 acc= 137 rate=0.856 tok/s=401.1
  long_code_review   pred= 192 draft= 168 acc= 134 rate=0.798 tok/s=379.2

Backend sampling: llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_M.gguf --spec-type draft-mtp --seed 42 -bs

python3 mtp-bench.py
  code_python        pred= 192 draft= 168 acc= 135 rate=0.804 tok/s=411.4
  code_cpp           pred= 192 draft= 190 acc= 127 rate=0.668 tok/s=369.2
  explain_concept    pred= 192 draft= 210 acc= 119 rate=0.567 tok/s=330.2
  summarize          pred= 192 draft= 169 acc= 134 rate=0.793 tok/s=412.3
  qa_factual         pred= 192 draft= 170 acc= 134 rate=0.788 tok/s=414.4
  translation        pred= 192 draft= 176 acc= 132 rate=0.750 tok/s=400.4
  creative_short     pred= 192 draft= 207 acc= 120 rate=0.580 tok/s=337.8
  stepwise_math      pred= 192 draft= 160 acc= 137 rate=0.856 tok/s=437.1
  long_code_review   pred= 192 draft= 168 acc= 134 rate=0.798 tok/s=410.2

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes, paired with Codex.