feat(stt): add stt context options

[rosetta-livekit-bot]

Summary

Ports livekit/agents#6475 to agents-js as the upper stack on PR #2057:

• Adds sttContextOptions with keyterms, keytermDetection, and forwardChatContext.
• Keeps deprecated keytermsOptions mapped to the new STT context shape.
• Makes inference chat-context capability derive solely from supported models.
• Gates conversation forwarding at the session level with forwardChatContext.
• Preserves AssemblyAI's deprecated plugin carryover option for compatibility while validating and truncating provider context safely.
• Uses a patch changeset for @livekit/agents.

Source diff coverage

Coverage table

Source file	Classification
examples/voice_agents/basic_agent.py	Ported to examples/src/basic_agent.ts: renamed keytermsOptions to sttContextOptions in the matching example.
livekit-agents/livekit/agents/__init__.py and voice/__init__.py	Adapted through the existing JS root/voice exports; STTContextOptions is exported publicly.
livekit-agents/livekit/agents/inference/stt.py	Adapted to agents/src/inference/stt.ts: the lower stack's temporary inference carryover flag is removed; capability follows AssemblyAI U3 Pro model support and refreshes on model changes.
livekit-agents/livekit/agents/voice/agent_activity.py	Ported to agents/src/voice/agent_activity.ts: forwarding is gated by sttContextOptions.forwardChatContext, with listener cleanup.
livekit-agents/livekit/agents/voice/agent_session.py and keyterm_detection.py	Ported to the JS session/resolver files with deprecated mapping, defaults, and precedence behavior.
livekit-plugins/.../assemblyai/stt.py	Adapted to plugins/assemblyai/src/stt.ts: U3 Pro carryover, deprecated plugin compatibility warnings, 1,750-character validation, automatic tail truncation, and failure atomicity.
Source tests	Adapted across inference STT, AgentSession/activity, keyterm resolution, and AssemblyAI plugin test suites, including model-derived capability, opt-out, truncation, validation, transitions, and cleanup.

No source diff files were omitted.

Validation

• Base: interim-prevue-pate / PR Support AssemblyAI inference context carryover #2057 at 0eebde463c7ab84705ff644327b77bf566de5758.
• Exact head: b5c231d283cae2d23136238062878d331e37e9fa.
• Focused tests passed: 154 passed, 2 skipped.
• Full build passed: 38/38 tasks.
• Isolated package lint passed with no errors and existing warnings; the normal linked-worktree command encountered duplicate ESLint plugin discovery.
• Focused Prettier check passed.
• Independent spec-compliance review approved.
• Independent code-quality review approved.

Cue voice E2E

Forwarding session: sid_95490732184e.

• Exact head: b5c231d283cae2d23136238062878d331e37e9fa.
• Mode: Cue voice with real LiveKit transport, voice injection/recording, and Cartesia sonic-3 through LiveKit inference TTS.
• Controlled boundary: a protocol-compatible AssemblyAI /v3/ws boundary captured the actual plugin query and UpdateConfiguration messages while supplying deterministic STT events; the LLM was deterministic.
• Capability/request shaping: universal-3-5-pro was model-derived supported while an unsupported-model probe was false. The initial request carried agent_context=ORIGINAL-2058; a 2,250-character assistant item produced exactly the final 1,750 D characters, SHA-256 d3078ec32af005bf146a63d2e11424b912726627eb5c22ecfa02fba92b2cd8f9.
• Validation atomicity: constructor and active-stream 1,751-character values were rejected before provider mutation; the prior valid context remained the last provider update.
• Cleanup/continuity: listener counts changed 1/1 to 0/0, and a second voice turn committed CONTINUITY-FORWARD-2058 with audio.

Opt-out session: sid_e98d8b0c8667.

• Model capability remained true while providerContextUpdateCount=0, proving forwardChatContext: false suppresses forwarding without changing capability.
• A second voice turn committed CONTINUITY-OPTOUT-2058 in the same session.

Artifacts:

• Both sessions' events.jsonl, both command result.json files per session, session WAVs, and all command WAVs under ~/.cue-cli/sessions/{sid_95490732184e,sid_e98d8b0c8667}/ were flushed and inspected directly.
• Limitation: the STT boundary was protocol-compatible rather than the production AssemblyAI service; LiveKit transport and inference TTS were real.

Known validation gaps

• Standard Turbo lint is unavailable in the nested worktree because ESLint discovers duplicate plugin installations; the isolated equivalent passed.

---

Ported from livekit/agents#6475

Original PR description

Adds stt_context_options on AgentSession — a single conversation-aware STT option grouping static keyterms, LLM keyterm_detection, and a forward_chat_context toggle that forwards agent replies to STTs that consume context natively (e.g. AssemblyAI Universal-3 Pro).

forward_chat_context is on by default and is the only control for carryover, applied consistently across the inference STT and the AssemblyAI plugin. Agent replies are now sent to AssemblyAI u3-pro by default when used via AgentSession; opt out with stt_context_options={"forward_chat_context": False}.

Deprecated (still work, warn):

• AgentSession(keyterms_options=...) → use stt_context_options
• AssemblyAI plugin agent_context_carryover → use stt_context_options.forward_chat_context (pass False to opt out)

Removes the unpublished agent_context_carryover param from the inference STT.

[changeset-bot]

🦋 Changeset detected

Latest commit: b5c231d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 37 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-anthropic	Patch
@livekit/agents-plugin-azure	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-cerebras	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-did	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-fishaudio	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-hume	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-liveavatar	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-minimax	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-mistralai	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-perplexity	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-runway	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugin-soniox	Patch
@livekit/agents-plugin-tavus	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 2 additional findings.