Quantized Qwen3-TTS on CPU: Benchmarks and the Graviton Cost Breakdown
A scored comparison of six CPU-quantized Qwen3-TTS runtimes, how quantized Qwen3-TTS stacks up against Kokoro on speed and quality, real-time-factor numbers measured on 4 vCPU/15GB hardware, and the AWS infra move (Fargate to Batch Spot Graviton) that cut TTS hosting to ~$5-7/mo.
Fast, high-quality text-to-speech on CPU-only hardware is now viable for podcast-length synthesis. Quantized Qwen3-TTS running on AWS Graviton Spot instances produces output quality comparable to GPU inference, at roughly $5-7/mo in compute.
Library comparison
Six CPU-targeting Qwen3-TTS runtimes, scored on license, model sizes, documented CPU real-time-factor (RTF), and cloning support:
| Rank | Candidate | Score | License | Sizes | CPU RTF (documented) | "ryan" preset | Transcript cloning |
|---|---|---|---|---|---|---|---|
| 1 | gabriele-mastrapasqua/qwen3-tts | 9 | MIT | 0.6B + 1.7B | real, multi-CPU (0.6B int8 sub-realtime; 1.7B int8 ~1.8-2.4x) | yes (default) | audio-only + optional .qvoice ICL text |
| 2 | Serveurperso/qwentts.cpp | 8 | MIT+Apache-2.0 | 0.6B + 1.7B | none (hardcoded thread /2, needs source patch) | yes | first-class --ref-text ICL |
| 3 | predict-woo/qwen3-tts.cpp | 5 | NONE (open issue) | 0.6B only | 1.94x (Ryzen 5 3600, 4T), ~3GB RAM | no | no |
| 4 | koboldcpp | 4 | AGPL-3.0 | 0.6B+1.7B | none (Vulkan only) | not found | via predict-woo underneath |
| 5 | HaujetZhao/Qwen3-TTS-GGUF | 5* | MIT | LLM-portion only | none | - | - |
| - | Danmoreng/qwen3-tts(.cpp) | 2 | - | - | - | - | - |
gabriele-mastrapasqua/qwen3-tts wins on the only criterion that matters for production: real, reproducible CPU RTF numbers across multiple CPUs, MIT-licensed, both model sizes, with a runtime --int8/--int4 quant switch and a --serve persistent HTTP mode (no reload between requests).
RTF numbers
Measured in-sandbox on 4 vCPU / 15GB, gabriele's 0.6B model:
| Mode | Quant | RTF |
|---|---|---|
| Non-cloning ("ryan" preset) | int8 | 3.07x |
| Non-cloning ("aiden" preset) | int8 | 2.85x |
| Non-cloning ("ryan" preset) | int4 | 3.26x |
| Cloning (reference audio) | int8 | 3.17x |
| Cloning (bootstrap-ref) | int8 | 2.50x |
RTF here means seconds of compute per second of audio - lower is faster. int4 was slower than int8 (dequantization overhead outweighs the smaller weights), so int8 is the shipped precision everywhere.
The production budget was ≤2.67x (fit a 45-minute episode in under 2 hours); only the bootstrap-ref case cleared that in the sandbox. On real Graviton hardware in production, throughput clears the budget comfortably. For reference, vendor-published numbers put Neoverse-N1 (Graviton2) at RTF ≈1.28x - that figure is upstream-claimed, not independently measured here.
Hardware and cost
TTS synthesis moved off ECS Fargate (x86) onto AWS Batch running EC2-Spot Graviton instances (c7g/c8g, arm64). The real bottleneck on Fargate wasn't raw compute - it was the model reloading from disk on every chunk, ~85-90 times per episode for a 2.35GB int8 model. A persistent --serve HTTP server eliminated the reload entirely.
Batch's EC2-Spot pool is what unlocks Graviton instance selection (Fargate has no arm64 Spot option with this flexibility). The Batch compute environment lists both c7g.4xlarge/8xlarge/12xlarge (Graviton3) and c8g.4xlarge/8xlarge/12xlarge (Graviton4), so Spot's capacity-optimized allocator can land on whichever pool has room. The binary is compiled with -mcpu=neoverse-v1 - the Graviton3 ISA, forward-compatible with Graviton4, but not neoverse-v2, which SIGILLs on c7g hardware.
A GPU tier existed briefly for a batched-inference experiment and has since been retired entirely - no CUDA dependency remains anywhere in the pipeline. Current steady-state cost for the Batch Spot Graviton TTS pipeline (bursty preprocessing + synthesis jobs, Spot-capacity-optimized): ~$5-7/mo.
Kokoro vs. quantized Qwen3-TTS
Kokoro is the CPU-TTS posterboy - tiny (82M params), Apache-licensed, and by far the fastest thing out there: ~5x real-time on a 32-core CPU. But it doesn't hold up on quality. My verdict after listening to both side by side: "Kokoro is terrible TTS quality, but blazing fast." Quantized Qwen3-TTS flips that - even the int4 build came back "incredible quality" on the same listening pass. The tradeoff is throughput, not fidelity.
| Kokoro | Qwen3-TTS int8 | Qwen3-TTS int4 | |
|---|---|---|---|
| Params | 82M | 0.6B | 0.6B |
| CPU speed | ~5x real-time (32-core CPU) | ~0.33x real-time (RTF 3.07x, shared 4-vCPU box); near real-time on ARM/Apple-Silicon-class cores (Neoverse-N1 RTF ≈1.28x) | ~0.31x real-time (RTF 3.26x) - slower than int8, not faster |
| Quality (my own listening test) | "terrible" | "incredible" | "incredible - even int4" |
| Voices / languages | 54 voices / 8 languages | 9 voices / 10 languages | 9 voices / 10 languages |
| Voice cloning | No | Yes (audio reference, or .qvoice text) | Yes |
| License | Apache 2.0 | MIT | MIT |
Those speed numbers come from different hardware - Kokoro's ~5x on a 32-core desktop CPU, Qwen's ~0.33x on a shared 4-vCPU cloud box - so don't read them as a clean ratio. What they do show: Kokoro wins raw throughput by roughly an order of magnitude, but Qwen closes most of that gap on ARM/Apple-Silicon-class cores, and even on modest x86 hardware it's already fast enough for podcast-length synthesis - the whole premise of this post.
If you're picking a model for your own hardware: int8 is the sweet spot - int4 isn't faster (dequant overhead outweighs the smaller weights on CPU) and only marginally smaller, so there's little reason to reach for it. Between sizes, 0.6B is the fast/light default and 1.7B trades some speed for higher quality if your hardware or budget allows. Either way, peak RAM stays around ~3.2GB, comfortably inside an 8GB laptop, and ARM/Apple-Silicon-class cores run noticeably faster than x86 for this workload.
Where it stands today
0.6B-int8 is the standard voice model for all users, free and paid. int8 runs everywhere; int4 was evaluated and abandoned as slower on Graviton's i8mm path. Output quality at int8 - and even int4 - is on par with what the GPU pipeline produced, at a fraction of the cost. Voice cloning is live today as a paid unlock, and the larger 1.7B model ships as a premium, show-level tier - not an automatic upgrade for paid accounts; paid standard voices still run the same 0.6B model as free.
For the full model-selection breakdown (presets vs. cloning vs. voice design), see the voice cloning guide.
Hear the quality yourself
My free podcast tool runs this same quantized Qwen3-TTS pipeline - fast CPU voices that don't sound like typical CPU TTS, at a fraction of GPU hosting cost.
