qwentts.cpp Lab

A collection of shell scripts and Gradio UIs for experimenting with qwentts.cpp — a C++ text-to-speech engine powered by the Qwen-Talker model.

Other UIs: Voice Design (TTS) · Custom Voice · Codec · Clone · Chunk Clone

Prerequisites

• A local build of qwentts.cpp — tested against 95b4840
• Python 3 virtual environment (venv/) with gradio>=4.0.0 and python-dotenv
• ffmpeg (required by chunk-clone.sh for concatenating chunks)
• (optional) google-perftools (libtcmalloc) preloaded by launch scripts for allocation performance

Setup

Copy .env.example to .env and edit it:

cp .env.example .env

At minimum, set QWENTTS_PATH to your qwentts.cpp build directory. All other values are sensible defaults.

Quick Start

Launch any Gradio UI with its corresponding launch-*.sh:

Mode	Launch Script	UI Module	Port
Base	./launch-base.sh	ui-base.py	7861
Voice Design (TTS)	./launch-tts.sh	ui-tts.py	7862
Custom Voice	./launch-customvoice.sh	ui-customvoice.py	7863
Codec (Embeddings)	./launch-codec.sh	ui-codec.py	7864
Clone	./launch-clone.sh	ui-clone.py	7860
Chunk Clone	./launch-chunk-clone.sh	ui-chunk-clone.py	7860

Then open http://127.0.0.1:<port> in your browser.

Shell Scripts

Core TTS Drivers

These scripts invoke the qwentts.cpp binaries directly. Model paths and inference flags are read from .env (see Configuration).

Script	.env Model Var	What It Does
base.sh	QWENTTS_BASE_MODEL	Plain TTS — synthesize text with no voice reference or instruction. Usage: base.sh [prompt.txt] [language]
tts.sh	QWENTTS_VOICEDSIGN_MODEL	Voice Design — describe the desired voice with an instruction string (e.g. "male, warm tone, moderate pace"; see docs/tts-voicedesign-instructions.md for all dimensions). Usage: tts.sh [prompt.txt] [language] [instruct]
customvoice.sh	QWENTTS_CUSTOMVOICE_MODEL	Named Speaker — pick from built-in voices (vivian, ryan, serena, etc.). Usage: customvoice.sh [prompt.txt] [language] [speaker]
clone.sh	QWENTTS_BASE_MODEL	Voice Cloning — clone a voice from a reference WAV + transcript text, or from pre-encoded .spk/.rvq embeddings. Usage: clone.sh [ref.wav] [ref.txt] [prompt.txt] [language]
chunk-clone.sh	QWENTTS_BASE_MODEL	Chunked Voice Cloning — splits a long prompt into sentence-bounded chunks, synthesizes each with cloning, then concatenates via ffmpeg. Requires embeddings/<voice>.spk, .rvq, and .txt (transcript). Usage: chunk-clone.sh [voice] [prompt.txt] [language]

Codec

Script	What It Does
codec.sh	Extracts speaker embeddings (.spk) and RVQ codes (.rvq) from a reference WAV, or encodes/decodes .rvq files. Outputs go to embeddings/. Mode controlled by QWENTTS_CODEC_MODE (extract, encode, decode). ~20 seconds of clean speech is enough for a good clone. Usage: codec.sh [input.wav] [output_dir] [speaker_name]

Server

Script	What It Does
server.sh	Starts the tts-server HTTP API (host/port from .env: QWENTTS_SERVER_HOST, QWENTTS_SERVER_PORT; defaults 127.0.0.1:8000) with auto-detect language. Model and codec are read from .env. Usage: server.sh [--base|--customvoice] — defaults to customvoice (built-in named speakers); --base switches to the base model for embedding-based voice cloning via POST /v1/voices.

Server API

OpenAI-compatible TTS endpoints (base URL http://127.0.0.1:8000):

Endpoint	What It Does
POST /v1/audio/speech	Synthesize text (OAI request format)
GET /v1/voices	List model speakers plus registered clone voices
POST /v1/voices	Register a clone voice (--base only): {name, ref_text, wav_b64} extracts server-side, or {name, ref_text, spk_b64, rvq_b64} takes pre-extracted latents verbatim
DELETE /v1/voices/{name}	Drop a registered voice
GET /health	Liveness probe

Synthesize — only input is required. response_format "pcm" (default) streams s16le 24 kHz mono as it is generated; "wav" returns a one-shot RIFF file:

# one-shot WAV file
curl -s http://127.0.0.1:8000/v1/audio/speech \
    -H 'Content-Type: application/json' \
    -d '{"input": "Hello world", "voice": "vivian", "response_format": "wav"}' \
    -o out.wav

# streaming raw PCM (play with ffmpeg, already a prerequisite)
curl -sN http://127.0.0.1:8000/v1/audio/speech \
    -H 'Content-Type: application/json' \
    -d '{"input": "Hello world", "voice": "vivian"}' | \
    ffplay -f s16le -ar 24000 -ac 1 -

Register a clone voice (--base only) — from a reference WAV + transcript, extracted server-side:

curl -s http://127.0.0.1:8000/v1/voices \
    -H 'Content-Type: application/json' \
    -d "{\"name\": \"myvoice\", \"ref_text\": \"$(cat ref.txt)\", \"wav_b64\": \"$(base64 -w0 ref.wav)\"}"

# then synthesize with it
curl -s http://127.0.0.1:8000/v1/audio/speech \
    -H 'Content-Type: application/json' \
    -d '{"input": "Hello", "voice": "myvoice", "response_format": "wav"}' \
    -o out.wav

Optional sampling overrides in the speech body (same names as the .env flags): seed, max_new_tokens, top_k, temperature, top_p, repetition_penalty.

Launchers

Each launch-*.sh sets up the environment (terminal title, LD_PRELOAD for tcmalloc) and runs the corresponding Python UI inside the virtual environment.

Python UIs

All UIs are built with Gradio and act as frontends to their matching shell scripts. They share a common set of advanced options:

• Seed — reproducibility control (-1 = random)
• Temperature / Top-k / Top-p — sampling parameters
• Repetition Penalty — discourages repeated tokens
• Sub-quantizer Temp — codec sub-quantizer temperature
• Output Format — wav16, wav24, or wav32
• Max New Tokens — generation length limit
• Stream by Line — incremental streaming
• Disable Flash Attention / Clamp FP16 — performance toggles

UI Details

Module	Backed By	Key Feature
ui-base.py	base.sh	Minimal TTS — text + language only
ui-tts.py	tts.sh	Voice design via natural-language instruction
ui-customvoice.py	customvoice.sh	Dropdown to select from built-in named speakers
ui-codec.py	codec.sh	Upload WAV → extract .spk + .rvq embeddings; lists saved embeddings (~20 s of audio is sufficient)
ui-clone.py	clone.sh	Two tabs: (1) upload reference WAV + transcript, (2) pick a pre-encoded voice from embeddings/
ui-chunk-clone.py	chunk-clone.sh	Upload a long prompt file and reference transcript; the script splits the prompt, synthesizes per chunk with voice cloning, and concatenates the result

Directory Layout

├── .env              # local config (git-ignored) — copy from .env.example
├── .env.example      # template with defaults and documentation
├── *.sh              # shell scripts (drivers + launchers)
├── docs/             # reference notes (e.g. voice-design instruction guide)
├── ui-*.py           # Gradio UI modules
├── launch-*.sh       # environment setup + UI launchers
├── embeddings/       # stored .spk, .rvq, .txt (transcript), .wav voice profiles (~20 s audio sufficient)
├── outputs/          # chunked synthesis results
├── voices/           # reference voice samples
├── prompts/          # prompt text files
├── venv/             # Python virtual environment
└── requirements.txt  # gradio>=4.0.0, python-dotenv

Configuration

All shared settings live in .env (git-ignored). A template is tracked as .env.example.

.env Sections

Section	Variables	Purpose
Paths	QWENTTS_PATH	Root of your qwentts.cpp build — scripts resolve binaries and models under this path.
Model Filenames	QWENTTS_BASE_MODEL, QWENTTS_VOICEDSIGN_MODEL, QWENTTS_CUSTOMVOICE_MODEL, QWENTTS_CODEC_MODEL, QWENTTS_TALKER_MODEL	Model files resolved as $QWENTTS_PATH/models/$VAR. Change these to swap quantizations (e.g. BF16 → Q4_K_M) without touching any script.
Defaults	LANG, SPEAKER, INSTRUCT	Fallback values for language, named speaker, and voice-design instruction. Override per-invocation via CLI args. Note: LANG must be a recognized language name (e.g. French, English) not a locale like fr_FR.UTF-8.
Inference Flags	QWENTTS_SEED, QWENTTS_TEMP, QWENTTS_TOP_K, QWENTTS_TOP_P, QWENTTS_REP_PEN, QWENTTS_SUB_TEMP, QWENTTS_FORMAT, QWENTTS_MAX_NEW	Shared defaults for all TTS scripts. Override per-command with QWENTTS_TEMP=0.5 ./tts.sh … or from the Gradio UI advanced options.
Optional Flags	QWENTTS_GREEDY, QWENTTS_STREAM, QWENTTS_NO_FA, QWENTTS_CLAMP_FP16	Leave empty to disable, set to any value to enable.
Codec	QWENTTS_CODEC_MODE	Default mode for codec.sh: extract, encode, or decode.
Server	QWENTTS_SERVER_HOST, QWENTTS_SERVER_PORT	Bind address and port for server.sh (defaults 127.0.0.1 / 8000).

Why .env

A plain env file was chosen over JSON/YAML because shell scripts already use ${VAR:-default} syntax, and per-command overrides (QWENTTS_TEMP=0.5 ./tts.sh) work naturally. Python UIs load it via python-dotenv.