🟢 Beginner–Intermediate ⚙️ Type: Text-to-Speech (TTS) / Voice Cloning 💸 Free & Open Source ⭐ Trending on GitHub
What is VoxCPM?
VoxCPM (specifically the massive new VoxCPM2 release) is a groundbreaking open-source Text-to-Speech (TTS) engine developed by OpenBMB and ModelBest. Unlike almost every other major TTS model on the market that chops audio into discrete “tokens,” VoxCPM is completely tokenizer-free. It utilizes an end-to-end diffusion-autoregressive architecture built on the powerful MiniCPM-4 language model backbone.
By modeling audio in a continuous space, VoxCPM eliminates the robotic stuttering and pronunciation artifacts that plague token-based systems. The newest 2-Billion parameter version is trained on over 2 million hours of speech data and natively supports 30 global languages (including 9 specific Chinese regional dialects) without requiring any manual language tags.
However, its true superpower is its unprecedented control. VoxCPM introduces Voice Design—the ability to conjure an entirely new human voice simply by describing it in natural language (e.g., “A young female surfer, speaking with a laid-back, chill, nasal drawl”). It also features Controllable Voice Cloning, allowing you to clone a real person’s voice but completely alter their emotion, speed, and delivery via text instructions.
Who is it for?
- Game Developers & Animators who need to generate hundreds of unique voices for NPCs without hiring voice actors. You can just type descriptions of the characters to generate consistent vocal identities.
- Content Creators & Audiobook Producers looking for highly expressive, emotional AI narration that supports seamless language switching in a single sentence.
- AI Application Builders creating virtual streamers or voice assistants who want to run a local, zero-latency inference engine without paying cloud API fees.
- Researchers & Hobbyists exploring the absolute cutting edge of diffusion-based zero-shot voice cloning.
What makes it special?
- Voice Design (No Audio Required) — You don’t need reference audio to make a custom voice. You just wrap a description in parentheses at the start of your prompt:
(A wise old man speaking very slowly and dramatically) Welcome, traveler.The AI invents the voice instantly. - Controllable Voice Cloning — Even when cloning a 3-second audio clip of your own voice, you retain full control. You can instruct the model to make your cloned voice speak angrily, whisper, or speak a completely different language fluently.
- Ultimate Cloning Mode — If you want a perfectly faithful, 1:1 replica of a voice with every breath and nuance matched, you simply provide the reference audio alongside its exact text transcript. The model uses “audio continuation” for maximum fidelity.
- 30-Language Multilingualism — You can feed it a paragraph containing English, Japanese, French, and a regional Chinese dialect, and it will transition between them perfectly in the same breath.
Requirements before you start
VoxCPM is a heavy, 2-billion-parameter neural network. To run it locally with optimal performance, you need capable hardware:
- NVIDIA GPU — An RTX 3090, 4090, or professional datacenter GPU is highly recommended for low real-time factor (RTF) generation.
- Python 3.10 to 3.12 — (Python 3.13 is not yet supported).
- PyTorch 2.5.0+ and CUDA 12.0+ — Required to run the highly optimized attention backends.
- Git & Hugging Face CLI — For downloading the model weights.
Step-by-step installation
Step 1 — Clone the Repository and Install Dependencies
Open your terminal, create a fresh Python environment, and clone the official OpenBMB repository:
git clone https://github.com/OpenBMB/VoxCPM.git
cd VoxCPM
pip install -r requirements.txt
pip install soundfileStep 2 — Download the Model Weights
Use the Hugging Face command-line tool to pull the massive 2B parameter weights down to your local machine into a new pretrained_models directory:
huggingface-cli download openbmb/VoxCPM2 --local-dir ./pretrained_models/VoxCPM2Step 3 — Generate Speech via Voice Design
Create a Python file (e.g., generate.py) and paste the following code to test the Voice Design feature. We will instruct the model to invent a completely new voice just from text:
import soundfile as sf
from voxcpm import VoxCPM
# Initialize the model
model = VoxCPM.from_pretrained("./pretrained_models/VoxCPM2", load_denoiser=False)
# The instructions in parentheses dictate the voice style and emotion
target_text = "(A young woman with a sweet, gentle voice, speaking softly) Hello! Welcome to the new era of generative audio."
wav = model.generate(
text=target_text,
cfg_value=2.0, # Higher values adhere closer to the instruction
inference_timesteps=10 # Lower is faster, higher gives better audio quality
)
# Save the output to a WAV file
sf.write("voice_design_output.wav", wav, model.tts_model.sample_rate)
print("Saved successfully!")Step 4 — Controllable Voice Cloning
If you want to clone an existing voice but change its emotion, modify your generate() call to include a path to a short 5-second reference audio clip:
wav = model.generate(
text="(Slightly faster, very angry tone) I told you never to touch that button!",
reference_wav_path="./path_to_your_voice.wav",
cfg_value=2.0,
inference_timesteps=10
)Common errors and fixes
| Error | What it means | How to fix it |
|---|---|---|
| CUDA Out of Memory (OOM) during generation | The 2B parameter model requires significant VRAM, especially when processing long text blocks or running high inference timesteps. | Split your text into shorter, individual sentences before passing them to the generator. You can also explore community 4-bit quantized versions of the model if your GPU has less than 16GB of VRAM. |
| Cloned voice has severe background noise or hiss | The model is so accurate that it actually cloned the background room noise from your reference audio clip. | Enable the load_denoiser=True flag during model initialization to apply the built-in ZipEnhancer. This will automatically clean the reference audio before the cloning process begins. |
| Voice Design ignores the text instructions | The model is struggling to align the natural language prompt with the acoustic generation. | Increase the cfg_value (Classifier-Free Guidance) closer to 3.0 or 4.0 to force the model to adhere strictly to your instructions in the parentheses. Ensure the instructions are exactly at the beginning of the string inside (). |
Free vs Paid comparison
| Feature | VoxCPM2 (Local Open Source) | ElevenLabs / Hume AI (Cloud SaaS) |
|---|---|---|
| Usage Cost | 🟢 $0 (Run endlessly on your hardware) | 🔴 Monthly subscription + per-character usage fees |
| Voice Design from Text | ✅ Yes (Built-in via prompt instructions) | ✅ Yes (Dedicated Voice Design UI) |
| Data Privacy | 🟢 100% Local and secure | 🔴 Voice samples uploaded to corporate servers |
| Deployment Difficulty | ⚠️ Requires Python and strong GPUs | 🟢 Turn-key REST APIs and Web Apps |
Bottom line: VoxCPM2 is a massive milestone for open-source audio generation. The ability to creatively direct a synthetic voice’s emotion and tone using nothing but text tags inside parentheses brings true directorial control to local TTS. If you have the GPU VRAM to run it, VoxCPM completely removes the need to pay exorbitant monthly fees to platforms like ElevenLabs for highly expressive character generation.
Alternatives — 3 similar tools
1. Supertonic 3
If your hardware cannot handle a massive 2-Billion parameter PyTorch model, Supertonic 3 is your best alternative. Running on ONNX, it is a tiny ~400MB model that operates blazingly fast on standard CPUs. While it lacks the extreme zero-shot “Voice Design from text” capabilities of VoxCPM, it is incredibly efficient for embedding into edge devices and web browsers.
🔗 github.com/supertone-inc/supertonic
2. Kokoro TTS
Another featherweight champion of the open-source TTS world. Kokoro is famous for its staggeringly human-like English inflections despite its tiny parameter size. It requires almost no setup and is the current favorite for homelabbers adding voices to local applications, though it does not yet support the 30 global languages offered by VoxCPM.
🔗 huggingface.co/hexgrad/Kokoro-82M
3. XTTSv2 (Coqui)
The veteran heavyweight of local voice cloning. Before VoxCPM introduced its diffusion-autoregressive approach, XTTS was the industry standard for 3-second zero-shot cloning. It remains incredibly stable and widely integrated into hundreds of open-source projects, but VoxCPM is rapidly proving to have more natural emotional control.
🚀 Want more free AI tools like this?
We find, test, and write setup guides for the best free and open-source AI tools — so you don’t have to dig through GitHub yourself.Browse Free AI Tools at globalaiforce.com/shop →
📸 Follow us for daily AI tool tips and tutorials: instagram.com/globalaiforce