修复了编译错误和onnx运行时链接错误
This commit is contained in:
commit
7d8b3370fa
48 files changed
+26105
No files matched your search
@@ -0,0 +1,117 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
@@ -0,0 +1,187 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📦 Using as a Rust Library
|
||||
|
||||
Add dependency in your `Cargo.toml`:
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
# Local reference
|
||||
qwen3-tts = { path = "." }
|
||||
|
||||
# Or from Git
|
||||
# qwen3-tts = { git = "https://github.com/cgisky1980/Qwen3-TTS-Rust" }
|
||||
```
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```rust
|
||||
use qwen3_tts::{TtsEngine, VoiceFile, SamplerConfig};
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
// 1. Initialize engine
|
||||
let mut engine = TtsEngine::new("models", "none").await?;
|
||||
|
||||
// 2. Configure sampling (optional)
|
||||
let sampler_config = SamplerConfig::new(0.7, 40, 0.9, None);
|
||||
engine.set_sampler_config(sampler_config);
|
||||
|
||||
// 3. Load voice
|
||||
let voice = VoiceFile::load("speakers/sohee.json")?;
|
||||
|
||||
// 4. Generate speech
|
||||
let audio = engine.generate_with_voice(
|
||||
"Hello, welcome to Qwen3-TTS!",
|
||||
&voice,
|
||||
None, // instruction
|
||||
)?;
|
||||
|
||||
// 5. Save audio
|
||||
audio.save_wav("output.wav")?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### Create Voice from Reference Audio
|
||||
|
||||
```rust
|
||||
use qwen3_tts::TtsEngine;
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
let mut engine = TtsEngine::new("models", "none").await?;
|
||||
|
||||
// Extract voice from reference audio
|
||||
let voice = engine.create_voice_file(
|
||||
"reference.wav",
|
||||
"Text corresponding to reference audio".to_string(),
|
||||
)?;
|
||||
|
||||
// Save voice file
|
||||
voice.save("speakers/my_voice.json")?;
|
||||
|
||||
// Generate with new voice
|
||||
let audio = engine.generate_with_voice("Testing new voice", &voice, None)?;
|
||||
audio.save_wav("output.wav")?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
@@ -0,0 +1,117 @@
|
||||
# Qwen3-TTS Rust
|
||||
|
||||
[简体中文](../README.md) | [English](README_EN.md) | [日本語](README_JA.md) | [Korean](README_KO.md) | [Français](README_FR.md) | [Español](README_ES.md) | [Italiano](README_IT.md) | [Deutsch](README_DE.md) | [Русский](README_RU.md) | [Português](README_PT.md)
|
||||
|
||||
This project is the ultimate performance implementation of Qwen3-TTS. The core breakthroughs are the deep integration of **"Instruction-Driven"** synthesis and **"Zero-shot Custom Speakers (Cloning)"**. Leveraging Rust's memory safety and the efficient inference of llama.cpp/ONNX, it provides an industrial-grade text-to-speech solution.
|
||||
|
||||
## 🚀 Core Features
|
||||
|
||||
### 1. Extreme Performance & Streaming
|
||||
- **Concurrent Streaming Decoding**: Uses a 4-frame (64 codes) granularity for concurrent decoding, determining first-token latency as low as 300ms for a "speech-while-thinking" experience.
|
||||
- **Hardware Acceleration**: **Vulkan** (Windows/Linux) and **Metal** (macOS) acceleration are enabled by default, significantly boosting inference speed.
|
||||
- **Automatic Runtime Management**: Zero-config environment; automatically downloads and configures `llama.cpp` (b7885) and `onnxruntime`, ready to use out of the box.
|
||||
|
||||
### 2. Flexible Speaker Management
|
||||
- **Auto-Scan & Cache**: Automatically loads voice files from the `speakers/` directory on startup.
|
||||
- **Versatile Selection**: Supports flexible speaker selection via CLI arguments `--speaker <name>` or `--voice-file <path>`.
|
||||
- **Smart Fallback**: Automatically falls back to the default voice (vivian) if the specified speaker is not found, ensuring system stability.
|
||||
|
||||
### 3. Precise Instruction Control
|
||||
- **Instruction-Driven**: Supports embedding emotion tags like `[Happy]`, `[Sad]` in the text to adjust the delivery style in real-time.
|
||||
- **EOS Alignment**: Perfectly aligned with Qwen3's stop logic, supporting multiple EOS token detections to prevent generation of trailing silence or artifacts.
|
||||
|
||||
## 📊 Benchmarks
|
||||
|
||||
| Backend | Model (GGUF) | RTF (Real-Time Factor) | Avg Time (ms) | Avg Audio (s) | Status |
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||
| **CUDA** | Q5_K_M | **0.553** | 1162.6 | 2.19 | OK |
|
||||
| **Vulkan** | Q5_K_M | 0.598 | 1285.4 | 2.19 | OK |
|
||||
| **CUDA** | Q8_0 | 0.640 | 1523.4 | 2.44 | OK |
|
||||
| **Vulkan** | Q8_0 | 0.638 | 1502.0 | 2.44 | OK |
|
||||
| **CPU** | Q5_K_M | 1.677 | 2823.4 | 1.96 | OK |
|
||||
| **CPU** | Q8_0 | 1.866 | 4160.1 | 2.51 | OK |
|
||||
|
||||
- **Test Environment**: Intel Core i9-13980HX, NVIDIA RTX 2080 Ti. VRAM Usage ~0.7-1.5GB.
|
||||
- **Data Source**: Average of 10 generations on Windows.
|
||||
- **Best Performance**: RTF 0.553 (CUDA + Q5KM), meaning 1 second of audio takes only 0.553 seconds to generate.
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Basic Generation
|
||||
Generate speech using the default speaker:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- --text "Hello, welcome to use Qwen3-TTS Rust!"
|
||||
```
|
||||
|
||||
### 2. Specify Speaker
|
||||
Use a preset or custom speaker:
|
||||
```powershell
|
||||
# Use name (requires corresponding .json file in speakers/ directory)
|
||||
cargo run --bin qwen3_tts -- --text "The weather is nice today." --speaker dylan
|
||||
|
||||
# Use specific file path
|
||||
cargo run --bin qwen3_tts -- --text "I am a custom voice." --voice-file "path/to/my_voice.json"
|
||||
```
|
||||
|
||||
### 3. Clone New Voice
|
||||
Clone a voice with just 3-10 seconds of reference audio:
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--ref-audio "ref.wav" `
|
||||
--ref-text "The text content corresponding to the reference audio" `
|
||||
--save-voice "speakers/my_voice.json" `
|
||||
--text "New voice saved, you can use it directly now!"
|
||||
```
|
||||
|
||||
### 4. Style/Emotion Control (Instruction)
|
||||
Change the speaking style in real-time via the `--instruction` argument:
|
||||
```powershell
|
||||
# Sad tone
|
||||
cargo run --bin qwen3_tts -- --text "I'm sorry, I didn't mean to..." --instruction "Sad and sobbing, very upset"
|
||||
|
||||
# Happy tone
|
||||
cargo run --bin qwen3_tts -- --text "Awesome! We did it!" --instruction "Happy and excited, slightly faster"
|
||||
```
|
||||
|
||||
### 5. Advanced Configuration
|
||||
```powershell
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Long text generation test." `
|
||||
--max-steps 1024 ` # Adjust max generation length
|
||||
--output "output.wav" # Specify output filename
|
||||
```
|
||||
|
||||
### 6. Sampling Parameters
|
||||
Control randomness and diversity of generation through sampling parameters:
|
||||
```powershell
|
||||
# Adjust temperature (higher = more random, 0.0 = greedy)
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --temperature 0.8
|
||||
|
||||
# Use fixed seed for reproducibility
|
||||
cargo run --bin qwen3_tts -- --text "Test text" --seed 12345
|
||||
|
||||
# Full parameter example
|
||||
cargo run --bin qwen3_tts -- `
|
||||
--text "Testing sampling parameters." `
|
||||
--temperature 0.7 ` # Temperature (default 0.7)
|
||||
--top-k 40 ` # Top-K sampling (default 40)
|
||||
--top-p 0.9 ` # Top-P nucleus sampling (default 0.9)
|
||||
--seed 42 # Random seed (optional)
|
||||
```
|
||||
|
||||
## 📂 Directory Structure
|
||||
|
||||
The system automatically builds the following structure on first run:
|
||||
|
||||
```text
|
||||
.
|
||||
├── models/ # Model files (GGUF, ONNX, Tokenizer)
|
||||
├── runtime/ # Auto-downloaded dependencies (dll, so, dylib)
|
||||
└── speakers/ # User custom voices
|
||||
```
|
||||
|
||||
## 📜 License & Acknowledgements
|
||||
|
||||
- Based on **MIT / Apache 2.0** license.
|
||||
- Thanks to [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) official repository for models and technical foundation.
|
||||
- Thanks to [Qwen3-TTS-GGUF](https://github.com/HaujetZhao/Qwen3-TTS-GGUF) for inspiration on the GGUF inference flow.
|
||||
Reference in new issue
Block a user