fix(workflows): ask before recording a volunteered value - #6980
Merged
Conversation
…-asking GetNameTask and the three credit-card sub-tasks scan the conversation on enter before asking. GetEmailTask, GetPhoneNumberTask and GetAddressTask were left with a bare "ask the user to provide ...", so a caller who volunteers the value up front is asked for it a second time - the task sees it in chat_ctx and its update tool is callable on enter, but nothing tells it to look. Same wording as the tasks that already do this.
…resses The audio instructions listed five example addresses that restate the normalization rules below them. The one case only an example carried, a local part with no domain, becomes a rule.
The on_enter instruction led with the tool call and made asking the fallback. With nothing from the user in context yet, the model filled the call from whatever address-shaped text was nearby and the task completed before the user spoke. Asking is the default action; recording an address the user already stated is the exception.
u9g
force-pushed
the
jason/inline-task-capture-fixes
branch
from
August 25, 2026 21:38
d659df6 to
ea888c4
Compare
…sked for The simulated caller holds back any goal that provides a personal detail until the agent asks for it, and the agent only collects a detail the caller has named. An opening line that carries the value is held back with it, so both sides wait and the call stalls out. The two date of birth scenarios now open by naming the detail, with the value kept for the ask.
The card task tells the caller to read digits one at a time, so the model treats ten digits as a partial number and asks for the rest rather than passing it to update_card_number. That neither records the short number nor goes silent, which is the behavior under test; requiring the words "invalid" or "incomplete" failed it on phrasing. The caller gives all sixteen digits on the retry: "let me start again" read to the card task as a request to abandon the capture.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
u9g
force-pushed
the
jason/inline-task-capture-fixes
branch
from
August 25, 2026 21:56
ea888c4 to
d3c0bff
Compare
u9g
changed the title
chenghao-mou
approved these changes
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two fixes surfaced by the data-capture-sim CI job (
examples/data_capture_sim), which had been red on main since Aug 14. Split out of #6849, which now stacks on this branch. The related framework fix is #6981.1.
GetEmailTask/GetPhoneNumberTask/GetAddressTaskfabricate a value on enter (beta/workflows)The
on_enterinstruction added to stop re-asking for a volunteered value led with the tool call ("First scan the conversation - if an email was already given, use it viaupdate_email_address… Only ask fresh when…"). With nothing from the user in context, gpt-4.1 filled the call from whatever address-shaped text was nearby in the system prompt (john.doe@gmail.com,mikeb22@gmail.com) and the task completed before the user spoke.tests/test_workflows.py::test_collect_emailfailed 3/3 locally.Clause order is the variable, not detail: wordings that lead with asking and make recording the exception pass. A/B on gpt-4.1, 5 trials each, two scenarios:
Confirmed 20/20 before applying to all three tasks.
test_collect_emailpasses 3/3 in both modalities. The example address list in the email task's audio instructions is replaced by rules; the one behavior only an example carried (a local part with no domain) becomes a rule.2. data-capture-sim scenarios
Result
With both this and the framework fix: data-capture-sim 3/10 → 9/10 (the remaining failure: the sim user skipped its opening line and gave the number before entering the task).
evaluationgreen.