feat(agent): configurable content_response_policy for content-without-tool-call responses (#3992)#3997
Open
Code-weaver1 wants to merge 1 commit into
Open
Conversation
…-tool-call responses CONTENT responses (prose, no tool call) currently set FINISHED unconditionally, silently terminating weaker/local models that narrate a step before its tool call (OpenHands#3992). EMPTY responses, by contrast, get a corrective nudge. Add content_response_policy to AgentBase: 'finish' (default, unchanged) or 'nudge' (emit the message, send corrective feedback pointing at the finish tool, continue the loop). Runaway prose loops are bounded by conversation-level stuck detection, the same mechanism bounding the existing empty-response nudge. Fixes OpenHands#3992
Collaborator
|
[Automatic Post]: I have assigned @neubig as a reviewer based on the repository MAINTAINERS file. Thanks in advance for the help! This comment was posted by an AI agent (OpenHands) on behalf of the user. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HUMAN:
AGENT:
Investigated, implemented, and unit-tested by an AI agent (Claude Code) operating under
Solomon Okello's direction; every claim below is backed by the included tests.
Why
#3992:
ResponseDispatchMixindispatches asymmetrically — an EMPTY response gets acorrective nudge and the loop continues, but a CONTENT response (prose, no tool call)
sets
execution_status = FINISHEDunconditionally. Weaker/local models narrate a stepin prose before emitting its tool call, so they get silently terminated mid-task with
exit 0. Three earlier attempts to change the default (#1303, #1304, #1385) didn't merge —
plain content is a legitimate completion signal for strong models. So this PR follows
the issue's second suggested direction: make the policy configurable, default
unchanged.
Summary
AgentBasefieldcontent_response_policy: Literal["finish", "nudge"], default"finish"— zero behavior change unless opted in."nudge": the prose message is emitted (preserved in history), a correctivenudge is sent (pointing at the
finishtool as the explicit completion path), and theloop continues. Runaway prose loops are bounded by conversation-level stuck detection —
the same mechanism that already bounds the EMPTY-response nudge (per the existing
docstring on
_send_corrective_nudge)._send_corrective_nudgegains an optionaltextkwarg so the content-case feedbackcan differ from the empty-case feedback (the model did produce a message; telling it
otherwise would be false).
Issue Number
Fixes #3992
How to Test
Three new tests:
test_content_response_nudge_policy_continues_loop— prose turn under"nudge": notFINISHED, agent message preserved, nudge follows, nudge text points at
finishtest_content_response_default_policy_is_finish— guards the no-behavior-changecontract
test_finish_tool_still_finishes_under_nudge_policy— explicit completion still worksThe pre-existing
test_content_response_sets_finishedcontinues to pass unchanged,which is the other half of the compatibility guarantee.