workflows: spell the value back once the caller refuses a confirmation - #6990
Open
u9g wants to merge 3 commits into
Open
workflows: spell the value back once the caller refuses a confirmation#6990u9g wants to merge 3 commits into
u9g wants to merge 3 commits into
Conversation
…efused ReadBack picks the read-back instruction a Get*Task hands its model after recording a value: the natural form the first time, the spelled form on every later attempt. A task records a value again only when the caller did not confirm it, and a value that sounds like another cannot be told apart by hearing it once more. Not wired into any task yet.
GetEmailTask, GetNameTask, GetPhoneNumberTask, GetAddressTask and GetDOBTask read a recorded value back the same way however many times the caller refuses it. A caller whose name sounds like another spelling can never accept the read-back: the agent had shayne.cole@gmail.com right three times in one call and each time said it as a word, which is the same sound as shane, so the caller re-spelt it and the call ended without a booking. Each task now hands its read-back through ReadBack: natural on the first attempt, spelled on every later one. The card tasks stay out, their values are never read back. verify_spelling on GetNameTask keeps spelling from the first attempt.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
chenghao-mou
approved these changes
Aug 26, 2026
| natural="Repeat the address back to the user.", | ||
| spelled=( | ||
| f"Repeat the address field by field, spelling the street name letter by " | ||
| f"letter: {address_fields}" |
Member
There was a problem hiding this comment.
should we provide spaced input so they don't hallucinate letters, like the strawberry quiz?
|
|
||
| def instruction(self, *, natural: str, spelled: str) -> str: | ||
| self._attempts += 1 | ||
| return natural if self._attempts == 1 else spelled |
Member
There was a problem hiding this comment.
I wonder if we should go one step further like "A as in alpha" for the 3rd time.
theomonnom
reviewed
Aug 26, 2026
Comment on lines
+87
to
+100
| class ReadBack: | ||
| """Escalates a value's confirmation read-back once the first one is not accepted. | ||
|
|
||
| A task records the value again only when the caller did not confirm it, so the | ||
| second and later read-backs spell the value out instead of saying it naturally: | ||
| a value that sounds like another cannot be told apart by hearing it again. | ||
| """ | ||
|
|
||
| def __init__(self) -> None: | ||
| self._attempts = 0 | ||
|
|
||
| def instruction(self, *, natural: str, spelled: str) -> str: | ||
| self._attempts += 1 | ||
| return natural if self._attempts == 1 else spelled |
Member
There was a problem hiding this comment.
This utility seems very tiny? can't we just inline it?
A single bool per task replaces the ReadBack helper: once a value has been recorded, every later read-back of it is spelled. GetNameTask seeds the bool from verify_spelling, so its separate branch goes away.
theomonnom
approved these changes
Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GetEmailTask,GetNameTask,GetPhoneNumberTask,GetAddressTaskandGetDOBTaskread a recorded value back the same way however many times the caller refuses it. A caller whose name sounds like another spelling can never accept that read-back. In a hotel-receptionist simulation (staging runSR_yxUq2cWd3C3D, jobSRJ_8GbAvzvRUXAF) the agent hadshayne.cole@gmail.comright three separate times and each time said it as a word, which sounds identical to "shane"; the caller re-spelt it six times and the call ended without a booking.Change
ReadBack(beta/workflows/utils.py) picks the read-back instruction a task hands its model after recording a value: the natural form on the first attempt, the spelled form on every later one. A task records a value again only when the caller did not confirm it, so the second attempt is by definition after a refusal.Each of the five tasks threads its read-back through it:
verify_spelling=Truekeeps spelling from the first attempt)The card tasks are deliberately left out: their values are never read back.
Verification
tests/test_workflow_readback.py(unit, hermetic):ReadBackpicks natural then spelled; each task's second_update_*_implreturns a different instruction from its first and, where the value can be spelled, carries the separated form.GetEmailTaskonopenai/gpt-4.1, audio modality:"…shayne.cole@gmail.com. Is this correct?"×3"s h a y n e . c o l e @ g m a i l . c o m. Is this correct?"on the second and third.livekit.agents.beta.workflows) clean.Python only: agents-js has no workflow tasks.