Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 19 additions & 15 deletions examples/data_capture_sim/scenarios.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -141,14 +141,16 @@ scenarios:
- time of birth: 6:45 in the morning
- date of birth: March 3rd, 1992
DO, IN ORDER:
1. When asked for the date of birth, give ONLY the time first: "I was born
1. Open by naming the detail you are offering (your opening line). The
agent asks which detail you want to give; "date of birth" is the answer.
2. When asked for the date of birth, give ONLY the time first: "I was born
at 6:45 in the morning." Do not mention the date yet.
2. If the agent asks you to confirm before you've given a date, say "yes,
3. If the agent asks you to confirm before you've given a date, say "yes,
that's right" - do NOT volunteer the date until the agent explicitly
asks for the date itself.
3. When the agent asks for the date, give March 3rd, 1992.
4. Confirm the full date (and time) when read back.
5. Don't hang up until the agent reads back March 3rd, 1992 as captured.
4. When the agent asks for the date, give March 3rd, 1992.
5. Confirm the full date (and time) when read back.
6. Don't hang up until the agent reads back March 3rd, 1992 as captured.
agent_expectations: >
Records the 6:45 AM time of birth but recognizes the date is still
missing: if the user tries to confirm before any date exists, the agent
Expand All @@ -164,16 +166,18 @@ scenarios:
- label: Date of birth corrected while confirming
instructions: |
PERSONA: Ana Sofia Torres, mixes up day and month at first.
OPENING LINE: "Sure, my date of birth - it's July 9th, 1988."
OPENING LINE: "I'd like to give you my date of birth."
FACTS:
- first date given: July 9th, 1988
- corrected date: September 7th, 1988
DO, IN ORDER:
1. Give July 9th, 1988 when asked.
2. When the agent asks you to confirm, say: "hold on, I always flip those -
1. Open by naming the detail you are offering (your opening line). The
agent asks which detail you want to give; "date of birth" is the answer.
2. Give July 9th, 1988 when asked.
3. When the agent asks you to confirm, say: "hold on, I always flip those -
it's September 7th, 1988. Yes, that's right."
3. If the agent asks you to confirm the corrected date, confirm it.
4. Don't hang up until the agent reads back September 7th, 1988 as captured.
4. If the agent asks you to confirm the corrected date, confirm it.
5. Don't hang up until the agent reads back September 7th, 1988 as captured.
agent_expectations: >
After the correction at the confirmation step, the agent confirms and
completes with September 7th, 1988 — never July 9th. It responds verbally
Expand All @@ -196,16 +200,16 @@ scenarios:
DO, IN ORDER:
1. When the agent asks for your card number, give the 10-digit partial
number as if it were complete.
2. When the agent says it's invalid or asks you to repeat, give the full
4242 4242 4242 4242.
2. When the agent says it's invalid, incomplete, or asks you to continue or
repeat, give all sixteen digits: 4242 4242 4242 4242.
3. Repeat the full number when asked to confirm it.
4. Provide the expiration, security code, and name as asked, repeating
each back when asked to confirm.
5. Don't hang up until the agent confirms the card details are complete.
agent_expectations: >
Tells the user the number is invalid (wrong length) and asks them to
repeat it — it must not accept the 10-digit number, must not go silent
after the invalid attempt, and must not read the card number back aloud.
Does not record the 10-digit number, and keeps talking: it asks for the
full number or for the remaining digits. It must not go silent after the
short attempt and must not read the card number back aloud.
After the full 16-digit number is given and repeated, the flow proceeds
through expiration, security code, and cardholder name to completion.
tags:
Expand Down
7 changes: 6 additions & 1 deletion livekit-agents/livekit/agents/beta/workflows/address.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,12 @@ def __init__(
)

async def on_enter(self) -> None:
self.session.generate_reply(instructions="Ask the user to provide their address.")
self.session.generate_reply(
instructions=(
"Ask the user for their address. If the user already stated one earlier in "
"this conversation, record it with update_address instead of asking again."
)
)

def _build_update_address_tool(self) -> llm.FunctionTool:
# Built dynamically so we can apply IGNORE_ON_ENTER per-instance
Expand Down
15 changes: 8 additions & 7 deletions livekit-agents/livekit/agents/beta/workflows/email_address.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,12 @@ def __init__(
)

async def on_enter(self) -> None:
self.session.generate_reply(instructions="Ask the user to provide an email address.")
self.session.generate_reply(
instructions=(
"Ask the user for their email address. If the user already stated one earlier "
"in this conversation, record it with update_email_address instead of asking again."
)
)

def _build_update_email_tool(self) -> llm.FunctionTool:
# Built dynamically so we can apply IGNORE_ON_ENTER per-instance
Expand Down Expand Up @@ -162,16 +167,12 @@ def _confirmation_required(self, ctx: RunContext) -> bool:
PERSONA = "You are only a single step in a broader system, responsible solely for capturing an email address."

AUDIO_SPECIFIC = """\
Handle input as noisy voice transcription. Expect that users will say emails aloud with formats like:
- 'john dot doe at gmail dot com'
- 'susan underscore smith at yahoo dot co dot uk'
- 'dave dash b at protonmail dot com'
- 'jane at example' (partial—prompt for the domain)
- 'theo t h e o at livekit dot io' (name followed by spelling)
Handle input as noisy voice transcription. Users say emails aloud.
Normalize common spoken patterns silently:
- Convert words like 'dot', 'underscore', 'dash', 'plus' into symbols: `.`, `_`, `-`, `+`.
- Convert 'at' to `@`.
- Recognize patterns where users speak their name or a word, followed by spelling: e.g., 'john j o h n'.
- If only the part before the '@' is given, prompt for the domain.
- Filter out filler words or hesitations.
- Assume some spelling if contextually obvious (e.g. 'mike b two two' → mikeb22).
Don't mention corrections. Treat inputs as possibly imperfect but fix them silently."""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,12 @@ def __init__(
)

async def on_enter(self) -> None:
self.session.generate_reply(instructions="Ask the user to provide their phone number.")
self.session.generate_reply(
instructions=(
"Ask the user for their phone number. If the user already stated one earlier "
"in this conversation, record it with update_phone_number instead of asking again."
)
)

def _build_update_phone_number_tool(self) -> llm.FunctionTool:
# Built dynamically so we can apply IGNORE_ON_ENTER per-instance
Expand Down
Loading