fix(models): don't offset streamed content_index by the reasoning item#3757
fix(models): don't offset streamed content_index by the reasoning item#3757AmirF194 wants to merge 1 commit into
Conversation
The Chat Completions stream handler computed each assistant content part's content_index by counting the reasoning item, but reasoning is emitted as a separate output item (its own output_index), not a content part of the assistant message. So when a response streams reasoning followed by text (or a refusal) — e.g. reasoning models routed through the Chat Completions adapter such as deepseek-reasoner — the text part was streamed with content_index=1 while it lands at message.content[0] in the final message and in what the OpenAI Responses API emits for the same output. A consumer reconstructing the message via message.content[event.content_index] writes to the wrong slot. Drop the reasoning increment in both the text and refusal branches. The text<->refusal cross-increment (both are real content parts of the message) is unchanged, and the final message assembly appends parts in order without using these indices, so only the streamed events change. Regression test fails on main (content_index=1) and passes with the fix; the stream + reasoning suites stay green (62 passed).
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5e4d6e02fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # The reasoning item is a separate output item, so it must not shift | ||
| # this content index (see the text branch above). | ||
| refusal_index = 0 |
There was a problem hiding this comment.
Keep refusal indexes aligned when text arrives later
When the stream order is reasoning -> refusal -> text, removing the reasoning offset here makes the refusal part emit at content_index=0; the later text branch then emits text at content_index=1 because a refusal already exists. However response.completed is still assembled with text before refusal (assistant_msg.content.append(text) then append(refusal)), so raw stream consumers reconstruct the parts swapped relative to the final message. This affects exactly the refusal-before-text shape the handler already supports, now with a preceding reasoning item.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Thanks, I checked this and the text/refusal swap is pre-existing, not introduced here. This PR only removes the reasoning-item offset; the text <-> refusal cross-increment in both branches is untouched. In the reasoning -> refusal -> text order: on main refusal streams at content_index=1 and text at 2; on this branch refusal at 0 and text at 1. In both cases the streamed order is refusal-then-text while the final message assembles text-then-refusal (lines 1044-1047 append text before refusal unconditionally), so the parts are swapped relative to the stream either way. This change only shifts the absolute indices down by one; it doesn't create or widen that gap.
It's worth fixing, but it's a separate concern (the final-assembly order vs arrival order), so I've kept this PR to the reasoning offset. Happy to open a follow-up to align the both-text-and-refusal case if you'd like.
Problem
The Chat Completions stream handler builds each assistant content part's
content_indexby counting the reasoning item. But reasoning is emitted as a separate output item (its ownoutput_index, 0), not a content part of the assistant message:_reasoning_output_count/assistant_message_output_indexalready place the message at a lateroutput_indexbecause of it.So when a stream contains reasoning followed by assistant text (and/or a refusal), the message's first content part is streamed with
content_index=1, even though it lands atmessage.content[0]in the finalresponse.completedmessage, and even though the OpenAI Responses API emitscontent_index=0for the same output.content_indexis the part's position within that item'scontentarray;output_indexis what distinguishes items.This affects reasoning models routed through the Chat Completions adapter (for example
deepseek-reasoner, and OpenAI-compatible reasoning models) that also return visible text or a refusal in the same turn. A consumer that reconstructs the message from the raw streamed events viamessage.content[event.content_index]writes to the wrong slot (or hits anIndexError). The SDK's high-levelRunResultis unaffected because it reads the finalresponse.completeditem; the impact is on consumers of the raw streamed events (raw_response_event).Root cause
In
chatcmpl_stream_handler.py, the text and refusal branches both did:The text/refusal cross-increment is correct (both are content parts of the same message). The reasoning increment is the bug.
Fix
Drop the reasoning increment in both branches. The final message assembly appends parts in order and never used these
content_indexvalues, so only the streamed events change; the finalresponse.completedmessage is byte-identical.Verification
Clean Docker container (
python:3.12-slim,uv sync --frozen):test_stream_handler_content_index_excludes_reasoning_itemfails onmain(streamedcontent_index=1) and passes with the fix.tests/models/test_openai_chatcompletions_stream.pyandtests/models/test_reasoning_content.py: 62 passed. The existingcontent_index == 1test (refusal-then-text, no reasoning) still passes, via the refusal increment.ruff check,ruff format --check, andmypyon the changed files: clean.No public API change (parameter/field order and import paths are untouched). This corrects the streamed
content_indexvalue so it matches the final message and the Responses API.