# Qwen3-TTS Rust [简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md) This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution. ## 🚀 Core Features ### 1. Extreme Performance & Streaming - **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience. - **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed. - **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box. ### 2. Flexible Speaker Management - **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup. - **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker ` or `--voice-file `. - **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability. ### 3. Precise Instruction Control - **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time. - **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts. ## 📊 Benchmarks | Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status | | :--- | :--- | :--- | :--- | :--- | :--- | | **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK | | **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK | | **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK | | **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK | | **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK | | **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK | - **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB. - **Data Source**: Average of 10 generations on Windows. - **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate. ## 🛠️ Quick Start ### 1. Basic Generation Generate speech using the default speaker: ```powershell cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!" ``` ### 2. Specify Speaker Use a preset or custom speaker: ```powershell # Use name (requires corresponding .json file in speakers/ directory) cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan # Use specific file path cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json" ``` ### 3. Clone New Voice Clone a voice with just 3-10 seconds of reference audio: ```powershell cargo run --bin qwen3_tts -- ` --ref-audio "ref.wav" ` --ref-text "The text content corresponding to the reference audio" ` --save-voice "speakers/my_voice.json" ` --text "New voice saved, you can use it directly now!" ``` ### 4. Style/Emotion Control (Instruction) Change the speaking style in real-time via the `--instruction` argument: ```powershell # Sad tone cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset" # Happy tone cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster" ``` ### 5. Advanced Configuration ```powershell cargo run --bin qwen3_tts -- ` --text "Long text generation test." ` --max-steps 1024 ` # Adjust max generation length --output "output.wav" # Specify output filename ``` ### 6. Sampling Parameters Control randomness and diversity of generation through sampling parameters: ```powershell # Adjust temperature (higher = more random, 0.0 = greedy) cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8 # Use fixed seed for reproducibility cargo run --bin qwen3_tts -- --text "Test text" --seed 12345 # Full parameter example cargo run --bin qwen3_tts -- ` --text "Testing sampling parameters." ` --temperature 0.7 ` # Temperature (default 0.7) --top-k 40 ` # Top-K sampling (default 40) --top-p 0.9 ` # Top-P nucleus sampling (default 0.9) --seed 42 # Random seed (optional) ``` ## 📦 Using as a Rust Library Add dependency in your `Cargo.toml`: ```toml [dependencies] # Local reference qwen3-tts = { path = "." } # Or from Git # qwen3-tts = { git = "https://github.com/cgisky1980/Qwen3-TTS-Rust" } ``` ### Basic Usage ```rust use qwen3_tts::{TtsEngine, VoiceFile, SamplerConfig}; #[tokio::main] async fn main() -> Result<(), Box> { // 1. Initialize engine let mut engine = TtsEngine::new("models", "none").await?; // 2. Configure sampling (optional) let sampler_config = SamplerConfig::new(0.7, 40, 0.9, None); engine.set_sampler_config(sampler_config); // 3. Load voice let voice = VoiceFile::load("speakers/sohee.json")?; // 4. Generate speech let audio = engine.generate_with_voice( "Hello, welcome to Qwen3-TTS!", &voice, None, // instruction )?; // 5. Save audio audio.save_wav("output.wav")?; Ok(()) } ``` ### Create Voice from Reference Audio ```rust use qwen3_tts::TtsEngine; #[tokio::main] async fn main() -> Result<(), Box> { let mut engine = TtsEngine::new("models", "none").await?; // Extract voice from reference audio let voice = engine.create_voice_file( "reference.wav", "Text corresponding to reference audio".to_string(), )?; // Save voice file voice.save("speakers/my_voice.json")?; // Generate with new voice let audio = engine.generate_with_voice("Testing new voice", &voice, None)?; audio.save_wav("output.wav")?; Ok(()) } ``` ## 📂 Directory Structure The system automatically builds the following structure on first run: ```text . ├── models/ # Model files (GGUF, ONNX, Tokenizer) ├── runtime/ # Auto-downloaded dependencies (dll, so, dylib) └── speakers/ # User custom voices ``` ## 📜 License & Acknowledgements - Based on **MIT / Apache 2.0** license. - Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation. - Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.