Skip to content

feat(desktop): add pitch-preserving TTS playback speed - #3194

Open
johnmatthewtennant wants to merge 53 commits into
mainfrom
jtennant/tts-playback-speed
Open

feat(desktop): add pitch-preserving TTS playback speed#3194
johnmatthewtennant wants to merge 53 commits into
mainfrom
jtennant/tts-playback-speed

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Context

Generated speech currently plays at a fixed 1x rate. Rodio's speed control shifts pitch, while Pocket synthesis rate changes model generation rather than playback. Playback speed needs to remain independent from Pocket voice selection and preserve the existing per-chunk streaming path.

Summary

This adds a locally persisted generated-speech playback setting with pitch-preserving Signalsmith time stretching. The slider covers 0.5x to 2x, the editable value accepts 0.25x to 4x, and Reset restores 1x. The control lives in Voice settings but remains independent from Pocket synthesis and voice selection.

Changes

  • Processes each complete Pocket playback chunk with Signalsmith Stretch before the chunk is appended to rodio.
  • Keeps the fixed device lead-in outside time stretching and retains the existing post-DSP cancellation check.
  • Preserves streaming time-to-first-playback. The first synthesized chunk plays without waiting for the full response.
  • Uses a bit-exact 1x bypass, persists the global speed locally across restarts, and supports precise typed values beyond the slider range.
  • Serializes native persistence writes and coalesces optimistic UI updates without allowing stale saves to overwrite newer intent.
  • Preserves the parent Voice Settings label, accessibility behavior, and live-only speech routing.
  • Adds focused coverage for pitch, ordering, onset, latency, persistence, cancellation, and the production chunk boundary.
  • Adds ssstretch 0.1.0 and the Signalsmith DSP implementation under the MIT license. Both are bundled at build time; normal use performs no download.

Related issue

None found.

Testing

Manual validation in the combined daily-driver build covered the 0.5x to 2x slider, typed values across the 0.25x to 4x accepted range, stable voice pitch, immediate cancellation, Reset to 1x, and persistence after restart. The production DSP probe measured 60 ms of compensated lookahead and processing below 0.21% of realtime at the sampled speeds.

Screenshots

The Voice settings control combines the bounded slider, editable value, and Reset action.

Speech playback speed

Reviewer-reproducible examples

  1. Launch Buzz Desktop and open Settings > Voice.
  2. Set Speech playback speed to 0.5x and play a Pocket voice preview or start a huddle with generated speech. Confirm speech is slower without a lower voice pitch.
  3. Set the control to 2x and confirm the next generated speech is faster without a higher voice pitch.
  4. Type a precise value outside the slider range, such as 0.25x or 4x, and confirm it is accepted and applied.
  5. Select Reset and confirm the value returns to 1x.
  6. Set a non-default value, quit and reopen Buzz, and confirm the value persists.
  7. During active generated speech, stop or interrupt playback and confirm cancellation remains immediate.

For a repeatable DSP measurement, run:

CARGO_HOME=$PWD/.hermit/rust cargo run --offline --release \
  --manifest-path desktop/src-tauri/Cargo.toml \
  --example tts_speed_probe

Observed output:

0.75x: processing=20.59ms, 0.206% realtime, output=320000 samples, compensated DSP lookahead=60.0ms
1.25x: processing=12.42ms, 0.124% realtime, output=192000 samples, compensated DSP lookahead=60.0ms
1.50x: processing=10.18ms, 0.102% realtime, output=160000 samples, compensated DSP lookahead=60.0ms

Pocket still submits the first completed synthesis chunk immediately rather than rendering the full response.

johnmatthewtennant added a commit that referenced this pull request Jul 27, 2026
@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review July 27, 2026 19:47
@johnmatthewtennant
johnmatthewtennant requested a review from a team as a code owner July 27, 2026 19:47
@johnmatthewtennant
johnmatthewtennant marked this pull request as draft July 27, 2026 19:53
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from f62db77 to 8de8b63 Compare July 28, 2026 00:45
@johnmatthewtennant
johnmatthewtennant changed the base branch from main to jtennant/pocket-tts-settings-v1 July 28, 2026 00:45
johnmatthewtennant added a commit that referenced this pull request Jul 28, 2026
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from 5a78f33 to 6fca35f Compare July 28, 2026 02:10
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch from 4891a92 to c118f5a Compare July 28, 2026 04:48
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from 6fca35f to f78b065 Compare July 28, 2026 05:12

johnmatthewtennant commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

🤖 New review-intent sweep completed on the frozen stacked candidate.

  • timeless-comments: 0 findings
  • stacksmith: 0 findings
  • dead-code: 1 valid P3, fixed by narrowing playback-speed DSP imports and removing the unused re-export suppression
  • selective dead-code rerun on the patched candidate: 0 findings

Published DCO-signed head: a5139463d1ca15eb9ee8292568429fae2e610e34 (base remains 7e5cc4b60796a4ceb5dfa10629587417dbbff85e). Focused patched-content validation passed: Rust formatting check, 8 playback-speed tests, warning-denied Tauri clippy, and git diff --check. GitHub CI finished green with 18 passing and 7 skipped checks. PR remains open and draft; no readiness or merge action was taken.

Manual validation still outstanding: listen to Mary and Marius preview plus active-huddle speech at 0.75x, 1.25x, and 1.50x; verify pitch, first phoneme, tail, cancellation, persistence after restart, and Reset to 1.00x.

@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch 2 times, most recently from 7cc0804 to acb302e Compare July 29, 2026 00:24
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from a513946 to 5b74959 Compare July 29, 2026 14:20
@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review July 29, 2026 14:50
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from 5b74959 to 6aebd8f Compare July 29, 2026 15:01
John Tennant and others added 14 commits July 29, 2026 11:55
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
@johnmatthewtennant
johnmatthewtennant marked this pull request as draft July 29, 2026 16:42
Signed-off-by: John Tennant <jtennant@block.xyz>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/tts-playback-speed branch from 6aebd8f to 347ab73 Compare July 29, 2026 17:08
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review July 29, 2026 20:27
@johnmatthewtennant johnmatthewtennant added triage-ready Appropriate for agentic review and removed triage-ready Appropriate for agentic review labels Jul 29, 2026
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant

Copy link
Copy Markdown
Contributor Author

🤖 ### Speech playback speed

The slider covers 0.5x to 2x, the editable value accepts 0.25x to 4x, and Reset restores 1x.

pocket-voices

johnmatthewtennant added a commit that referenced this pull request Jul 30, 2026
Base automatically changed from jtennant/pocket-tts-settings-v1 to jt/buzz-voice-refactor July 31, 2026 11:07
npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d and others added 3 commits July 31, 2026 09:59
Restack after #3910 relanded #2467+#3208 onto main. Logical-base merge
(base 6186fc2, rerere disabled); tree matches the pre-verified
merge-tree prediction 799cc15.

Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Restack after #3910: brings #3259's content under this PR so the
3259×3194 interaction is resolved once, here. Hand-resolved the two
predicted conflicts (rerere disabled, logical base 6186fc2):

- huddle/tts_settings.rs preview_pocket_voice: keep #3259's
  pocket_voice_reference() resolution + #3194's playback_speed
  thread-through into TtsPipeline::new_with_voice.
- testing/e2eBridge.ts: union of #3259's import/delete-voice arms and
  #3194's get/set playback-speed arms; 245 unique case labels, zero
  duplicates (checked programmatically).

MERGE ORDER CONSTRAINT: #3259 must land before this PR.

Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
@tlongwell-block
tlongwell-block changed the base branch from jt/buzz-voice-refactor to main July 31, 2026 15:15
Retack after #3259 landed on main as squash c104eec. The squash
severed ancestry from this branch's previous merge parent (3259 head
0ef85c2), manufacturing phantom conflicts in exactly the two known
3259x3194 interaction files. Main's blobs for both files are
byte-identical to 3259's head blobs, so the resolution re-applies the
already-blessed union from e8b00d4 unchanged (verified by blob hash):

- huddle/tts_settings.rs: 099e6ad (3259 pocket_voice_reference +
  3194 playback_speed plumbing)
- testing/e2eBridge.ts: a58b760 (union arms, 245 unique labels)

Also brings in #3751 (avatars fix), previously verified zero overlap
with this PR's surface.

Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pitch-preserving speed control is useful, and the branch is now cleanly shaped against main (19 files, mergeable). Two correctness issues still need resolution before approval: the stretcher resets at hidden Pocket model-unit boundaries, and component-local persistence can discard the user's latest speed when Settings unmounts. Details are inline.

Cancellation is rechecked under the player-operation lock before append, so cancelled processed audio is rejected. The accessibility wiring also looks sound. The new target-wide MSVC CXXFLAGS affect every C++ dependency, so the current Windows CI result is important evidence for the final candidate; it was still running when I submitted this review.

Reviewed at exact head 9ff70fff53f4c9c21d061fbf6ce92838560dd89b.

— Carl, reviewing on Wes's behalf

Comment thread desktop/src-tauri/src/huddle/tts.rs Outdated
Comment thread desktop/src/features/settings/ui/playbackSpeedPersistence.ts Outdated
Signed-off-by: John Tennant <jtennant@squareup.com>
* origin/main:
  docs(nips): add single-coordinate manual-unread override layer and verification model to NIP-RS (#2864)
  chore(release): release Buzz Desktop version 0.5.3
  fix(release): make immutable desktop release operable (#3943)

Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants