phonon-31m-en GGUF

gradium/phonon-31m-en (Gradium's Phonon, English, 31M parameters) in more GGUF quantizations than the two the original repo ships. Made for Quick Inference.

The files follow the original's layout and load with Gradium's runtime (xn-ptts: the ptts command, Python, the browser package), picking the file with --weights and its format with --quant:

ptts --model hf://danyalmalik/phonon-31m-en-GGUF --weights model.q6k.gguf --quant q6k --lang en "Hello." -o hello.wav

Files

As in Gradium's own quantizations (made with their quantize example, the same way), only the flow model's 24 transformer linear layers are quantized; everything else is bfloat16, and the runtime keeps the Mimi decoder in float32. model.q8.gguf and model.q4k.gguf are byte-identical to the original repo's.

File	`--quant`	Size	Bits per weight (quantized layers)
`model.bf16.gguf`	`f32`	82.2 MB	16
`model.q8.gguf`	`q8_0`	64.5 MB	8.5
`model.q6k.gguf`	`q6k`	59.9 MB	6.56
`model.q5k.gguf`	`q5k`	57.4 MB	5.5
`model.q5_1.gguf`	`q5_1`	58.6 MB	6.0
`model.q5_0.gguf`	`q5_0`	57.4 MB	5.5
`model.q4k.gguf`	`q4k`	55.1 MB	4.5
`model.q4_1.gguf`	`q4_1`	56.3 MB	5.0
`model.q4_0.gguf`	`q4_0`	55.1 MB	4.5
`model.q3k.gguf`	`q3k`	52.6 MB	3.44

model.q3k.gguf does not load in Gradium's released runtime, which accepts formats down to q4k only. Its tensor library supports Q3_K, so it runs with the runtime's Quant list extended by Q3_K (a few lines in ptts/src/synth.rs), which is how it was tested below. All quantizations are calibration-free (no importance matrix). Q2_K was tried and left out: it makes the model lose the end of the sentence and produce no usable speech.

Measurements

8 English sentences (including a phone number, a date and an email address), seed 1234, temperature 0.3, 4 CPU threads, Gradium's runtime on a laptop CPU. Word error rate from Whisper small.en transcripts; the original's 6.8% is number formatting ("555 0142" transcribed as "555-0142"), not mistakes. Weight cosine compares the quantized layers with the original weights. RTF is generation time over audio length (lower is faster).

Format	Weight cosine	Peak memory	RTF	First audio	Length vs original	Word error rate
original (`model.safetensors`)	—	358 MB	0.053	28 ms	1.00	6.8%
bf16	1.0	342 MB	0.053	27 ms	1.00	6.8%
q8_0	0.99999	270 MB	0.043	19 ms	1.00	6.8%
q6k	0.9998	270 MB	0.048	38 ms	1.00	5.7%
q5k	0.9993	282 MB	0.048	36 ms	1.01	6.8%
q5_1	0.9992	285 MB	0.070	107 ms	1.00	6.8%
q5_0	0.9990	296 MB	0.082	140 ms	0.99	6.8%
q4k	0.9972	289 MB	0.053	40 ms	1.02	9.1%
q4_1	0.9967	293 MB	0.061	78 ms	1.03	6.8%
q4_0	0.9962	276 MB	0.049	37 ms	1.02	5.7%
q3k	0.9886	277 MB	0.050	41 ms	1.04	6.8%

These are modified files: the quantized layers' values differ from the original's (above).

Licence

Apache 2.0, as the original (see LICENSE). Phonon is by Gradium and builds on Kyutai's Pocket TTS and Mimi codec.

Loading…