fix(google): don't cut streaming TTS off at the connect timeout - #6986
Open
biztex wants to merge 1 commit into
Open
fix(google): don't cut streaming TTS off at the connect timeout#6986biztex wants to merge 1 commit into
biztex wants to merge 1 commit into
Conversation
SynthesizeStream passed conn_options.timeout (10s by default, documented
as the connect timeout) as the timeout of streaming_synthesize(). For a
gapic streaming method that value is the gRPC deadline of the whole bidi
call, and the request generator keeps the call open for as long as the
LLM produces text for the segment. Any reply that takes longer than 10s
to generate and synthesize was therefore ended with DeadlineExceeded
mid-utterance. Since audio had already been emitted the base class does
not retry, so the rest of the reply was dropped ("TTS failed after
partial audio was already sent to the user, skip retrying").
The deadline also left grpc's request poller parked on the sentence
stream, so the finally-block aclose() of the request generator raced
with it and failed with "aclose(): asynchronous generator is already
running" whenever the LLM was still producing text.
Open the RPC without a deadline and only bound the wait for the first
audio chunk with conn_options.timeout, mapping it to APITimeoutError as
before. Let grpc own the request generator's lifetime: it finalizes it
when the call completes or is cancelled, which is what happens on the
first-chunk timeout.
The tests drive tts.stream() against a fake client that mirrors grpc.aio
(background request poller, whole-call deadline, read cancellation
cancels the RPC): a reply produced over longer than the connect timeout
is synthesized in full, and a server that never answers still raises
APITimeoutError within the timeout.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
longcw
approved these changes
Aug 27, 2026
| except asyncio.TimeoutError: | ||
| raise APITimeoutError() from None | ||
|
|
||
| while resp is not None: |
Contributor
There was a problem hiding this comment.
nit: maybe pushing the first chunk and then async for resp in resp_iter?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
SynthesizeStream._run_streampassesconn_options.timeout(10 s by default, documented as the connect timeout) astimeout=ofstreaming_synthesize(). For a gapic streaming method that is the gRPC deadline of the whole bidi call (google.api_core→grpc.aio.StreamStreamMultiCallable(timeout=...), "duration of time in seconds to allow for the RPC"), and the request generator keeps the call open for as long as the LLM produces text for the segment. Any reply that takes longer than 10 s to generate and synthesize is ended withDeadlineExceeded→APITimeoutErrormid-utterance, and because audio was already emitted the base class doesn't retry ("TTS failed after partial audio was already sent to the user, skip retrying") — the rest of the turn is dropped.use_streaming=Trueis the default, so this hits every long reply.The deadline has a second effect: grpc only cancels its request-poller task in
cancel(), never when the deadline fires. So when the LLM is still producing text at that moment, the poller is mid-__anext__oninput_genand thefinally: await input_gen.aclose()fails withRuntimeError: aclose(): asynchronous generator is already running(the traceback in #2951), leaking the poller.Fix
stt.py'sstreaming_recognize), and useconn_options.timeoutonly to bound the wait for the first audio chunk (asyncio.wait_foron the firstanext), still surfacing it asAPITimeoutError. Subsequent chunks are unbounded, since the gaps between them follow the LLM's pace.aclose()that raced with grpc's poller is gone.ChunkedStreamis untouched: there the kwarg is a per-request deadline for a single unarysynthesize_speech, which is what it should be.Tests
tests/test_plugin_google_tts.pydrivestts.stream()against a fake client that mirrors grpc.aio (request iterator consumed by a background task,timeoutenforced as a whole-call deadline, cancelling a read cancels the RPC and its poller):test_streaming_not_cut_off_by_connect_timeout: three sentences pushed over ~0.45 s withtimeout=0.2— every sentence reaches the fake and every audio byte is delivered. Onmainit fails with theaclose()RuntimeError above and a leaked poller task.test_streaming_first_response_timeout: a server that never answers while the input is still open raisesAPITimeoutErrorwithin the timeout, with no leaked tasks.ruff / mypy (strict) clean.
Fixes #3347
Fixes the timeout part of #5117 (its chirp_3 voice-name error is a separate config/model-support question).
Also fixes the
aclose(): asynchronous generator is already runningfailure reported in #2951.