Skip to content

feat(boson-avatar): add avatar plugin - #7019

Open
warlockee wants to merge 11 commits into
livekit:mainfrom
warlockee:codex/boson-avatar-plugin
Open

feat(boson-avatar): add avatar plugin#7019
warlockee wants to merge 11 commits into
livekit:mainfrom
warlockee:codex/boson-avatar-plugin

Conversation

@warlockee

@warlockee warlockee commented Aug 27, 2026

Copy link
Copy Markdown

Summary

  • add livekit-plugins-boson-avatar, a BYOK Avatar plugin for Boson Higgs Audio-Driven Avatar
  • compose with the existing AgentSession voice pipeline instead of creating a second voice participant
  • accept provider connection settings from the host application; the plugin has no built-in provider hostname
  • add project-scoped Avatar discovery so applications can populate a face picker without hard-coded IDs
  • mint a least-privilege, room-scoped Avatar token and route generic 24 kHz agent PCM over LiveKit data streams
  • add provider start/delete lifecycle handling, idempotent retries, startup compensation, and cancellation-safe cleanup
  • register the package in the uv workspace and livekit-agents[boson-avatar] optional dependencies

Developer experience

The host application is the composition root: it supplies its Boson endpoint and API key, loads the project catalog server-side, and passes the Avatar selected by its UI to the session.

from livekit.plugins import boson_avatar

avatars = await boson_avatar.list_avatars()
# Return only avatar_id + name to the browser's face picker.

avatar = boson_avatar.AvatarSession(
    avatar_id=selected_avatar_id,
    api_url="https://your-boson-endpoint.example/v1/avatar/livekit",
)
await avatar.start(session, room=ctx.room)

api_url may alternatively be supplied through BOSON_AVATAR_API_URL; BOSON_API_KEY supplies the API key. The plugin intentionally does not define or assume a production hostname. It accepts the audio output of any compatible AgentSession voice pipeline and has no dependency on a Boson audio model. The URL must be an HTTPS base URL (loopback HTTP is allowed for local development); the plugin appends /avatars and the session resource paths. list_avatars() owns its short-lived HTTP client, so it also works in a normal application server outside a LiveKit job context.

Validation

  • uv run ruff format --check . and uv run ruff check . — passed
  • uv run python scripts/check_types.py — passed strict mypy
  • plugin unit suite — 27 tests plus 33 subtests passed, including catalog validation, endpoint validation, concurrent close, and repeated-cancellation regressions
  • wheel and sdist build passed
  • exact PR package completed a least-privilege live staging E2E: can_subscribe=False, Avatar joined, targeted PCM streamed, audio and video tracks published, and DELETE cleanup returned all four GPU workers to idle

Integration boundary

Deployment of a compatible Boson Avatar endpoint is an application/provider concern, not a plugin release dependency. The host app owns endpoint selection and composes the voice pipeline, Avatar plugin, LiveKit room, API credentials, and Avatar selection. The browser receives only catalog display data; the provider URL and Boson API key stay server-side.

The provider should retain its project-scoped catalog plus room-disconnect and max-duration reapers for process-kill and lost-response boundaries where a client can never receive the provider session ID.

@CLAassistant

CLAassistant commented Aug 27, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@warlockee
warlockee marked this pull request as ready for review August 27, 2026 18:18
@warlockee
warlockee requested a review from a team as a code owner August 27, 2026 18:18

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@warlockee
warlockee force-pushed the codex/boson-avatar-plugin branch from be915d5 to 30625f3 Compare August 28, 2026 04:13
@warlockee

warlockee commented Aug 28, 2026

Copy link
Copy Markdown
Author

Ready for maintainer review at 30625f3.

  • All GitHub CI checks, CLA, and Devin Review are green.
  • Addressed and resolved all three security review threads. Provider errors are no longer emitted through exception tracebacks; logs retain only a safe error type, and provider identifiers are marked as PII.
  • Proactively stripped sensitive provider exception context after exhausted retries and covered compensation, shutdown, background-task, and retry logging with regression tests.
  • Local validation: 30 Boson Avatar tests passed, full repository make check passed across 650 typed source files, and both wheel and sdist built successfully.
  • The AvatarSession remains audio-provider agnostic and has no Boson audio dependency or hard-coded api.boson.ai endpoint.

@tinalenguyen, would you be able to take a look? You reviewed the recent Runway and Protoface Avatar plugin contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants