Skip to content

common : add support for multiple end sequences in the reasoning budget sampler#25544

Open
aldehir wants to merge 8 commits into
ggml-org:masterfrom
aldehir:reasoning-budget-multi-seq
Open

common : add support for multiple end sequences in the reasoning budget sampler#25544
aldehir wants to merge 8 commits into
ggml-org:masterfrom
aldehir:reasoning-budget-multi-seq

Conversation

@aldehir

@aldehir aldehir commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Overview

Add support for multiple end sequences in the reasoning budget sampler. This is required to support #24202, which allows Qwen 3+ models to terminate their thinking process with either </think> or <tool_call>.

This PR contains quite a bit of changes to support this:

  • The trie/ac data structures in peg-parser.cpp are moved out to a separate trie.cpp/h. These are used to handle the token matching underneath, as they already support matching multiple sequences.
  • Refactored reasoning_end_tag to reasoning_end_tags.
  • Updated all parser implementations to return a list of tags, although currently they all only provide one.
  • When forcefully terminating reasoning, the first sequence is used. We could retain a single reasoning_end_tag and have an extra reasoning_additional_end_tags field if that feels more appropriate.
  • Added common_reasoning_budget_get_end_match() to obtain the matching sequence. Since <tool_call> is also a trigger token for the grammar sampler, we need to feed it to the sampler when grammar_lazy = true. Either this, or we revert our decision to suppress the grammar sampler while reasoning.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: Yes, to extract the data structures and write some tests.

@aldehir aldehir requested review from a team, ggerganov and pwilkin as code owners July 10, 2026 23:07
@github-actions github-actions Bot added testing Everything test related server labels Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

server testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant