fix(ai-gateway): preserve Gemini thought signatures - #5377
Conversation
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
| extra_body: CustomLlmExtraBodySchema.optional(), | ||
| remove_from_body: z.array(z.string()).optional(), | ||
| use_gemini_reasoning_transform: z.boolean().optional(), | ||
| reasoning_details_transform: ReasoningDetailsTransformSchema.optional(), |
There was a problem hiding this comment.
CRITICAL: Stored use_gemini_reasoning_transform configs are silently dropped
CustomLlmApiConfigSchema is a default Zod object, so unknown keys are stripped. Existing custom_llm2.definition and model_experiment_variant_version.upstream blobs still store use_gemini_reasoning_transform: true from #5217. safeParse succeeds, the flag is discarded, and reasoning_details_transform stays undefined.
buildDirectProvider then sets responseTransforms: null, disabling Gemini thought/signature mapping for every already-configured custom LLM and experiment. rotateApiKey also persists the stripped blob.
Map use_gemini_reasoning_transform: true → reasoning_details_transform: 'gemini-thought' on parse (and omit on false/absent).
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| details.push({ | ||
| type: ReasoningDetailType.Text, | ||
| text: value.content, | ||
| index: 0, |
There was a problem hiding this comment.
WARNING: Every Gemini reasoning block is emitted with index: 0
OpenRouter-style clients merge reasoning_details by index and concatenate data/text for the same index (see mergeReasoningDetail in the extension stream client). Thought text, a message-level signature, and every tool-call signature all use index: 0, so a single delta or assembled stream can collapse distinct encrypted blobs into data: sig1+sig2 or overwrite type.
Text chunks can share an index; each encrypted signature needs its own index (and must not share one with text).
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| continue; | ||
| } | ||
|
|
||
| const toolCall = |
There was a problem hiding this comment.
WARNING: Id-less encrypted details attach to the first tool call
The response transform emits message-level thought_signature as reasoning.encrypted with no id. On the inverse, a missing detail.id always selects toolCalls[0].
An assistant turn that has both a root signature and tool_calls (typical after a thought-then-function stream) therefore writes the message signature onto the first tool call instead of the message. Last write also overwrites any earlier signature on that call.
Only fall back to toolCalls[0] when the message has a single tool call and no separate message-level signature; otherwise attach id-less details to the message.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
| session_id: sessionId, | ||
| }); | ||
|
|
||
| applyReasoningDetailsTransform(provider, requestToMutate); |
There was a problem hiding this comment.
WARNING: reasoning_effort is now read after remove_from_body
applyReasoningDetailsTransform runs after transformRequest. For Gemini that is required so extra_body.google is merged first, but buildDirectProvider also deletes remove_from_body keys first.
Previously the Gemini path captured reasoning_effort before those deletes. A custom/experiment config with remove_from_body: ['reasoning_effort'] plus gemini-thought now always looks like “effort absent” (include_thoughts: true, no thinking_level).
Read reasoning_effort before applying remove_from_body, or exclude it from that deletion when the Gemini transform is enabled.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
Code Review SummaryStatus: 4 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)CRITICAL
WARNING
Files Reviewed (13 files)
Fix these issues in Kilo Cloud Reviewed by grok-4.6 · Input: 163.4K · Output: 29.2K · Cached: 831K Review guidance: REVIEW.md from base branch |
1 participant
Summary
reasoning_detailsgoogle-gemini-v1encrypted details to nativethought_signaturefields on follow-up requestsVerification
pnpm --filter web exec jest --runInBand --forceExit --runTestsByPath src/lib/ai-gateway/rewriteModelResponse.test.ts src/lib/ai-gateway/experiments/build-direct-provider.test.ts src/lib/ai-gateway/providers/apply-provider-specific-logic.test.ts src/lib/ai-gateway/providers/provider-definitions.test.ts 'src/app/api/openrouter/[...path]/route.test.ts'(180 tests passed)pnpm --filter web typecheckpnpm --filter web lintgit diff --check