Skip to content

fix: retry without tools when OpenAI-compatible server returns empty response - #13091

Open
thanhnnict wants to merge 1 commit into
continuedev:mainfrom
thanhnnict:fix/empty-response-retry
Open

fix: retry without tools when OpenAI-compatible server returns empty response#13091
thanhnnict wants to merge 1 commit into
continuedev:mainfrom
thanhnnict:fix/empty-response-retry

Conversation

@thanhnnict

Copy link
Copy Markdown

Problem

Some OpenAI-compatible servers (NIM, vLLM, TGI) return content: null with an empty tool_calls: [] array when tools are provided in the request but the model decides not to call any tool. This results in blank responses rendered in the Continue GUI.

Solution

Detect empty response (content: null + tool_calls: []) and retry the same request without the tools parameter. The model then responds with text content normally.

Behavior Matrix

Model Response tools in request Retry? Result
content: "Hello", tool_calls: [] Yes ❌ No Text shown normally
content: null, tool_calls: [{...}] Yes ❌ No Tool call executed
content: null, tool_calls: [] Yes ✅ Yes Retry → text shown
content: null, tool_calls: [] No ❌ No Empty (no tools to remove)

Affected Models

  • nvidia/nemotron-ultra-253b (NIM) — frequently
  • deepseek-ai/DeepSeek-V4-Flash (vLLM) — occasionally
  • Any OpenAI-compatible server with tool support

Trade-offs

  • Latency: Adds one extra API call when empty response is detected (~2-5s)
  • Correctness: Model genuinely wants to say nothing → forced to respond (acceptable for coding assistant)
  • Scope: Only affects chatCompletionNonStream (non-streaming path)

Related

…response

Some OpenAI-compatible servers (NIM, vLLM, TGI) return content: null
with an empty tool_calls array when tools are provided in the request
but the model decides not to call any tool. This results in blank
responses in the Continue GUI.

Fix: detect empty response (content: null + tool_calls: []) and retry
the same request without the tools parameter. The model then responds
with text content normally.

Affected models:
- nvidia/nemotron-ultra-253b (NIM) — frequently
- deepseek-ai/DeepSeek-V4-Flash (vLLM) — occasionally
- Any OpenAI-compatible server with tool support

Related: continuedev#5508
@thanhnnict
thanhnnict requested a review from a team as a code owner August 5, 2026 15:40
@thanhnnict
thanhnnict requested review from sestinj and removed request for a team August 5, 2026 15:40
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant