Qwen3-TTS Rust
简体中文 | English | 日本語 | Korean | Français | Español | Italiano | Deutsch | Русский | Português
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of "Instruction-Driven" synthesis and "Zero-shot Custom Speakers (Cloning)". Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
🚀 Core Features
1. Extreme Performance & Streaming
- Concurrent Streaming Decoding: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
- Hardware Acceleration: Vulkan (Windows/Linux) and Metal (macOS) acceleration are enabled by default, significantly boosting inference speed.
- Automatic Runtime Management: Zero-config environment; automatically downloads and configures
llama.cpp(b7885) andonnxruntime, ready to use out of the box.
2. Flexible Speaker Management
- Auto-Scan & Cache: Automatically loads voice files from the
speakers/directory on startup. - Versatile Selection: Supports flexible speaker selection via CLI arguments
--speaker <name>or--voice-file <path>. - Smart Fallback: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
3. Precise Instruction Control
- Instruction-Driven: Supports embedding emotion tags like
[Happy],[Sad]in the text to adjust the delivery style in real-time. - EOS Alignment: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
📊 Benchmarks
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|---|---|---|---|---|---|
| CUDA | Q5_K_M | 0.553 | 1162.6 | 2.19 | OK |
| Vulkan | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
| CUDA | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
| Vulkan | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
| CPU | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
| CPU | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
- Test Environment: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
- Data Source: Average of 10 generations on Windows.
- Best Performance: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
🛠️ Quick Start
1. Basic Generation
Generate speech using the default speaker:
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
2. Specify Speaker
Use a preset or custom speaker:
# Use name (requires corresponding .json file in speakers/ directory)
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
# Use specific file path
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
3. Clone New Voice
Clone a voice with just 3-10 seconds of reference audio:
cargo run --bin qwen3_tts -- `
--ref-audio "ref.wav" `
--ref-text "The text content corresponding to the reference audio" `
--save-voice "speakers/my_voice.json" `
--text "New voice saved, you can use it directly now!"
4. Style/Emotion Control (Instruction)
Change the speaking style in real-time via the --instruction argument:
# Sad tone
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
# Happy tone
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
5. Advanced Configuration
cargo run --bin qwen3_tts -- `
--text "Long text generation test." `
--max-steps 1024 ` # Adjust max generation length
--output "output.wav" # Specify output filename
6. Sampling Parameters
Control randomness and diversity of generation through sampling parameters:
# Adjust temperature (higher = more random, 0.0 = greedy)
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
# Use fixed seed for reproducibility
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
# Full parameter example
cargo run --bin qwen3_tts -- `
--text "Testing sampling parameters." `
--temperature 0.7 ` # Temperature (default 0.7)
--top-k 40 ` # Top-K sampling (default 40)
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
--seed 42 # Random seed (optional)
📦 Using as a Rust Library
Add dependency in your Cargo.toml:
[dependencies]
# Local reference
qwen3-tts = { path = "." }
# Or from Git
# qwen3-tts = { git = "https://github.com/cgisky1980/Qwen3-TTS-Rust" }
Basic Usage
use qwen3_tts::{TtsEngine, VoiceFile, SamplerConfig};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// 1. Initialize engine
let mut engine = TtsEngine::new("models", "none").await?;
// 2. Configure sampling (optional)
let sampler_config = SamplerConfig::new(0.7, 40, 0.9, None);
engine.set_sampler_config(sampler_config);
// 3. Load voice
let voice = VoiceFile::load("speakers/sohee.json")?;
// 4. Generate speech
let audio = engine.generate_with_voice(
"Hello, welcome to Qwen3-TTS!",
&voice,
None, // instruction
)?;
// 5. Save audio
audio.save_wav("output.wav")?;
Ok(())
}
Create Voice from Reference Audio
use qwen3_tts::TtsEngine;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut engine = TtsEngine::new("models", "none").await?;
// Extract voice from reference audio
let voice = engine.create_voice_file(
"reference.wav",
"Text corresponding to reference audio".to_string(),
)?;
// Save voice file
voice.save("speakers/my_voice.json")?;
// Generate with new voice
let audio = engine.generate_with_voice("Testing new voice", &voice, None)?;
audio.save_wav("output.wav")?;
Ok(())
}
📂 Directory Structure
The system automatically builds the following structure on first run:
.
├── models/ # Model files (GGUF, ONNX, Tokenizer)
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
└── speakers/ # User custom voices
📜 License & Acknowledgements
- Based on MIT / Apache 2.0 license.
- Thanks to Qwen3-TTS official repository for models and technical foundation.
- Thanks to Qwen3-TTS-GGUF for inspiration on the GGUF inference flow.