fix(agents): make mergeFrames linear instead of quadratic#2081
Open
adaro wants to merge 1 commit into
Open
Conversation
mergeFrames rebuilt the accumulated buffer with a boxed spread on every frame (new Int16Array([...data, ...frame.data])), costing O(total_samples * frame_count) copies. Merging a 30s utterance of 10ms frames (k=3000) blocked the event loop for ~60s; it now sums lengths in one pass and copies each frame once into a preallocated Int16Array. Hot callers: the Silero VAD inference loop (twice per 32ms window), the STT plugins' recognize paths (whole-utterance buffers), and TTS ChunkedStream.collect(). Fixes livekit#1995 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: b0d5a26 The changes in this PR will be included in the next version bump. This PR includes changesets to release 38 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #1995.
mergeFramesinagents/src/utils.tsrebuilt the accumulated buffer with a boxed spread on every iteration (data = new Int16Array([...data, ...frame.data])), so merging k frames of S total samples cost O(S*k) element copies with per-element boxing instead of O(S). This function sits on several hot paths: the Silero VAD inference loop calls it twice per 32ms window, all the buffer-based STT plugins'recognizepaths pass whole-utterance frame arrays, and TTSChunkedStream.collect()merges every synthesized frame.Measured on the repro from the issue (10ms 48kHz mono frames, median of repeated runs):
Before the fix, ratios grow quadratically (N=500 is ~2600x N=10); after, growth is linear and a 30s utterance merge drops from ~62s of blocked event loop to 0.22ms.
Changes Made
mergeFramesnow validates and sums frame lengths in one pass, then copies each frame once into a preallocatedInt16Arrayviaset()— same approach as the local helper already inagents/src/inference/vad.ts. Semantics are unchanged: sameTypeErrors for empty/mismatched buffers, always returns a fresh frame for array input, passthrough for single-frame input.mergeFrames(ordering, multi-channel, error cases) — there was no existing coverage.@livekit/agentspatch).Pre-Review Checklist
Testing
pnpm build(tsup +tsc --declaration),eslint,prettier --check, and the full@livekit/agentsvitest suite (1336 passed) all pass locally. Note:src/telemetry/traces.otel2.type.test.tsfails to import on a clean checkout ofmaintoo (missing optional otel2 dep, see #1879) — unrelated to this change.Additional Notes
@livekit/rtc-node'scombineAudioFrameshas the same shape of problem (buffer.map((x) => [...x.data]).flat()— linear but heavily boxed), which is why this PR fixesmergeFramesin place rather than delegating to it.🤖 Generated with Claude Code