Skip to content

fix(voice): unwedge scheduler when reply is interrupted before playout starts#2070

Open
Raysharr wants to merge 1 commit into
livekit:mainfrom
Raysharr:fix/pre-playout-interrupt-wedge
Open

fix(voice): unwedge scheduler when reply is interrupted before playout starts#2070
Raysharr wants to merge 1 commit into
livekit:mainfrom
Raysharr:fix/pre-playout-interrupt-wedge

Conversation

@Raysharr

Copy link
Copy Markdown
Contributor

Fixes #2065

Problem

When a pipeline reply is interrupted before its playout has started, AgentActivity.mainTask hangs forever on the interrupted handle's generation future and _currentSpeech is never cleared — the agent goes permanently silent for the rest of the session, until an external mechanism (idle/duration guard, user disconnect) tears it down.

Mechanism (full trace in #2065): the reply task's interrupted branch awaits raceWithAbort(audioOutput.waitForPlayout(), replyAbortController.signal). The playback-finished event never fires because playback never began, and replyAbortController.abort() only happens in code that runs after the segment loop returns — the same loop that is blocked on that await. The reply task therefore never settles the generation future, and mainTask's second wait on it (ThrowsPromise.race([generation, abortFuture.await])) deadlocks, since abortFuture only resolves on activity teardown.

Change

In mainTask's interrupted-generation branch:

  • bound the wait on the generation future (INTERRUPTED_GENERATION_TIMEOUT, 3 s) in addition to abortFuture;
  • if the generation still hasn't settled (tracked by a settled flag, so a concurrent settle never triggers the fallback) and the activity isn't shutting down, log one warning and cancel the handle's pending tasks via the existing cancelAndWait util — cancelling the reply task trips its own replyAbortController, so the parked raceWithAbort resolves and the reply task runs its normal interrupted-reply cleanup (partial-transcript commit, speaking→listening transition, _markGenerationDone());
  • _markDone() only as a safety net if the tasks don't settle within REPLY_TASK_CANCEL_TIMEOUT.

Healthy paths are untouched: an uninterrupted reply never enters this branch; a post-playout-start interruption settles the generation within milliseconds (the timer is cleared in finally, and the settled flag prevents any force-abort); activity shutdown still exits via abortFuture / signal.aborted.

Validation

  • New regression test agents/src/voice/agent_activity_pre_playout_interrupt.test.ts: an audio output that accepts frames but never reports playback started/finished, plus a non-force interrupt fired on the first captured frame. Without the fix the test hangs until timeout; with the fix the wedged handle settles (~3 s) and a follow-up reply reaches the output.
  • Full agents package suite passes (101 files / 1290 tests); typecheck, throws:check, and lint on the touched files are clean.
  • Downstream validation in the production-shaped voice harness that originally reproduced AgentSession: reply interrupted before playout start wedges _currentSpeech forever (1.5.3) #2065 (multi-agent handoff with user speech overlapping the handoff window, VAD turn detection): the trigger scenario went from a deterministic 0/3 to 3/3 with these exact wait-bound + force-abort semantics applied as a dist-level patch, with the agent recovering its voice ~5 s after the interrupt and no recurrence of the wedge signature.

@Raysharr
Raysharr requested a review from a team as a code owner July 19, 2026 17:53
@changeset-bot

changeset-bot Bot commented Jul 19, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 6663341

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 37 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-azure Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AgentSession: reply interrupted before playout start wedges _currentSpeech forever (1.5.3)

1 participant