feat(soniox): speed, cloned voice, and word timestamps for TTS#6312
feat(soniox): speed, cloned voice, and word timestamps for TTS#6312mihafabcic-soniox wants to merge 3 commits into
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| def _is_spaceless_language(language: str) -> bool: | ||
| """Languages written without spaces (zh/ja): each character is its own word.""" | ||
| return language.split("-")[0].lower() in ("zh", "ja") |
There was a problem hiding this comment.
🚩 Spaceless language list only covers Chinese and Japanese
The _is_spaceless_language function at livekit-plugins/livekit-plugins-soniox/livekit/plugins/soniox/tts.py:369-371 only checks for zh and ja. Other spaceless or partially-spaceless languages like Thai (th), Lao (lo), Khmer (km), and Burmese (my) are not covered. If Soniox supports these languages, their character-level timestamps would be grouped into space-delimited words, which may produce incorrect results since these scripts don't use spaces between words. Worth verifying which languages Soniox TTS actually supports.
Was this helpful? React with 👍 or 👎 to provide feedback.
tinalenguyen
left a comment
tinalenguyen
left a comment
There was a problem hiding this comment.
hi, thanks for the PR! could you also address these issues:
-
Punctuation is deleted for Chinese/Japanese (tts.py:396, confirmed)
The zh/ja path keeps only alphanumeric characters. "好的，OK。" becomes "好的OK" — and because this text also becomes the chat-history message, the corrupted version gets fed back to
the LLM every turn. -
Only a plain space counts as a word boundary (tts.py:402, confirmed)
Newlines and tabs glue words together ("point.\nSecond" is one word with the wrong timestamp), and double spaces get collapsed, so the transcript no longer matches what the LLM said. -
Both of the above have one fix: use the shared tokenizer (tts.py:369, confirmed)
ElevenLabs solves this exact problem with tokenize.basic.split_words(split_character=True), which handles all whitespace, keeps punctuation, and covers CJK and Thai. The PR's
hardcoded ("zh", "ja") list means Thai — which Soniox supports — gets no incremental transcript at all (everything arrives as one giant word at the end).
| if stream.partial: | ||
| text, start = stream.partial | ||
| stream.emitter.push_timed_transcript( | ||
| TimedString(text=text, start_time=start) |
There was a problem hiding this comment.
are we able to pass an end_time?
2 participants
Adds three parameters to the Soniox TTS plugin:
speed: speech rate multiplier (0.7-1.3), sent in the per-stream config.voice: now also accepts a cloned-voice UUID (the server returns a per-stream error for a voice not in the project).return_timestamps(defaulttrue): emits word-aligned transcript deltas so the context reflects only what was spoken on interruption. Disable withreturn_timestamps=False.