VoxCPM2: Self-Host Your Own AI Voice Generator — Multilingual TTS with Voice Cloning
VoxCPM2: Self-Host Your Own AI Voice Generator
Voice synthesis has come a long way. What once required expensive cloud APIs and hours of studio recordings can now be done with a single open-source model running on your own hardware. Meet VoxCPM2 — a 2-billion-parameter, tokenizer-free Text-to-Speech system from OpenBMB (Tsinghua University) that generates studio-quality 48kHz audio in 30 languages, with support for voice cloning, voice design from natural-language descriptions, and real-time streaming.
Unlike traditional TTS models that discretize speech into tokens (losing subtle acoustic details), VoxCPM2 works directly in continuous latent space using a diffusion autoregressive architecture. The result is highly natural, expressive speech that preserves vocal nuances — timbre, rhythm, emotion, and pacing.
Why It's Trending
VoxCPM2 hit #1 on GitHub Trending in April 2026 and has already amassed over 30,000 stars. It's the first fully open-source TTS model to combine:
- Voice Design — create a synthetic voice from a text description alone (e.g., "a warm middle-aged man, slightly deep voice, calm tone")
- Controllable Voice Cloning — clone any voice from a short audio clip, then steer its emotion, pace, and style
- 30 languages in a single model — no per-language fine-tuning needed
- 48kHz output — true studio quality, rivaling proprietary systems like ElevenLabs and OpenAI TTS
- Apache 2.0 license — free for commercial use
Architecture Overview
VoxCPM2 follows a four-stage generative pipeline operating entirely in continuous latent space, bypassing discrete tokenization entirely:
-
Local Encoder — Groups consecutive audio frames into patches and encodes them into compact local representations, reducing sequence length for the language model.
-
Text-Semantic LM (TSLM) — A causal language model built on MiniCPM-4 that jointly processes text tokens and audio embeddings. This stage handles "what to say" — planning prosody, pacing, and emphasis from text content.
-
Residual Acoustic LM (RALM) — Fuses semantic and acoustic information using concatenation + projection (improved from VoxCPM 1.x's additive fusion). This bridges the gap between high-level planning and fine-grained audio details.
-
Local DiT (CFM) — A Conditional Flow Matching diffusion transformer that generates continuous audio latents at each autoregressive step. Uses multi-token conditioning (vs. single-token in VoxCPM 1.x) for richer expressiveness.
The generated latents are decoded by AudioVAE V2 — an asymmetric codec that accepts 16kHz reference audio and outputs 48kHz waveforms with built-in super-resolution.
For voice cloning, VoxCPM2 adds a structurally isolated reference-audio channel that separates timbre reference from continuation context. The Voice Design path accepts parenthetical descriptions (e.g., "(A young woman, gentle voice)Hello!") embedded in the input text.
Prerequisites
- A GPU with at least 8 GB VRAM (NVIDIA RTX 3090/4090 recommended)
- CUDA 12.0+ and PyTorch 2.5+
- Python 3.10 – 3.12
- 12 GB free disk space for the model weights
You can also run on CPU or Apple Silicon (MPS) using --device auto, but generation will be significantly slower.
Installation
Install the package with pip:
pip install voxcpmThe first time you import VoxCPM, it downloads the pretrained weights from Hugging Face (~5 GB):
# The weights are downloaded automatically on first use
# But you can pre-download them:
python -c "from voxcpm import VoxCPM; model = VoxCPM.from_pretrained('openbmb/VoxCPM2', load_denoiser=False)"Quick Start
Text-to-Speech
Create a Python script and run it:
from voxcpm import VoxCPM
import soundfile as sf
model = VoxCPM.from_pretrained(
"openbmb/VoxCPM2",
load_denoiser=False,
)
wav = model.generate(
text="VoxCPM2 brings studio-quality multilingual speech synthesis to everyone.",
cfg_value=2.0,
inference_timesteps=10,
)
sf.write("output.wav", wav, model.tts_model.sample_rate)
print("Saved: output.wav")The cfg_value controls how closely the model follows the text (higher = more strict), and inference_timesteps trades quality for speed (10 is a good default, 20 gives higher quality).
Voice Design (No Reference Audio)
Create a voice from nothing but a description:
wav = model.generate(
text="(A young woman, gentle and sweet voice)Hello, welcome to VoxCPM2!",
cfg_value=2.0,
inference_timesteps=10,
)
sf.write("voice_design.wav", wav, model.tts_model.sample_rate)The parenthetical description at the start controls all vocal characteristics: gender, age, tone, emotion, pace.
Voice Cloning
Clone a voice from a reference audio file:
wav = model.generate(
text="This is a cloned voice generated by VoxCPM2.",
reference_wav_path="path/to/voice.wav",
)
sf.write("clone.wav", wav, model.tts_model.sample_rate)
# With style control
wav = model.generate(
text="(slightly faster, cheerful tone)This is a cloned voice with style control.",
reference_wav_path="path/to/voice.wav",
cfg_value=2.0,
inference_timesteps=10,
)
sf.write("controllable_clone.wav", wav, model.tts_model.sample_rate)Streaming Generation
For real-time applications:
import numpy as np
chunks = []
for chunk in model.generate_streaming(
text="Streaming text to speech is easy with VoxCPM!",
):
chunks.append(chunk)
wav = np.concatenate(chunks)
sf.write("streaming.wav", wav, 48000)CLI Usage
VoxCPM also provides a command-line interface:
# Text-to-speech
voxcpm tts --text "Hello world" --output hello.wav
# Voice design
voxcpm design --text "Hello world" --control "Young female voice, warm and gentle" --output out.wav
# Voice cloning
voxcpm clone --text "This is a clone" --reference-audio path/to/voice.wav --output out.wav
# Batch processing
voxcpm batch --input texts.txt --output-dir outputs/Web Demo
VoxCPM ships with a built-in Gradio web interface:
python app.py --port 8808Open http://localhost:8808 in your browser for an interactive playground.
Production Deployment (Optional)
Nano-vLLM (High-Throughput)
For lower latency and concurrent requests:
pip install nano-vllm-voxcpmfrom nanovllm_voxcpm import VoxCPM
import numpy as np, soundfile as sf
server = VoxCPM.from_pretrained(model="/path/to/VoxCPM2", devices=[0])
chunks = list(server.generate(target_text="Hello from VoxCPM!"))
sf.write("out.wav", np.concatenate(chunks), 48000)
server.stop()Achieves RTF as low as ~0.13 on RTX 4090 (vs ~0.30 with standard PyTorch).
vLLM-Omni (Production Multi-Tenant)
For OpenAI-compatible API serving:
vllm serve openbmb/VoxCPM2 --omni --port 8000
# Then call it like OpenAI TTS:
curl http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"openbmb/VoxCPM2","input":"Hello from VoxCPM2!","voice":"default"}' \
--output out.wavModel Comparison
| Feature | VoxCPM2 | VoxCPM 1.5 | VoxCPM 0.5B |
|---|---|---|---|
| Parameters | 2B | 0.6B | 0.5B |
| Languages | 30 | 2 (zh, en) | 2 (zh, en) |
| Sample Rate | 48kHz | 44.1kHz | 16kHz |
| Voice Design | ✅ | — | — |
| Controllable Cloning | ✅ | — | — |
| VRAM (minimum) | ~8 GB | ~6 GB | ~5 GB |
| RTF (RTX 4090) | ~0.30 | ~0.15 | ~0.17 |
| License | Apache 2.0 | Apache 2.0 | Apache 2.0 |
VoxCPM2 is the recommended version for most use cases. Use VoxCPM 1.5 for CPU-only or memory-constrained environments.
Verification Checklist
pip install voxcpmcompletes successfully- Model loads:
VoxCPM.from_pretrained("openbmb/VoxCPM2")returns without errors - Basic TTS generates a
.wavfile that plays correctly - Voice design creates recognizable speech from a text description
- Voice cloning from a reference audio preserves the original speaker's timbre
- CLI commands produce output files
- Web demo (
python app.py) starts and is accessible on port 8808
Resources
- GitHub: github.com/OpenBMB/VoxCPM
- Documentation: voxcpm.readthedocs.io
- Live Demo: huggingface.co/spaces/OpenBMB/VoxCPM-Demo
- Model Weights: huggingface.co/openbmb/VoxCPM2
- Technical Report: arxiv.org/abs/2606.06928
- Nano-vLLM Engine: github.com/a710128/nanovllm-voxcpm
- vLLM-Omni: github.com/vllm-project/vllm-omni