Granite Speech 5.0 TurboCTC 470M — GGUF

GGUF files of IBM's granite-speech-5.0-470m-turboctc at revision 1dd23f46, the October 21, 2026 multilingual release: English, French, German, Spanish, Portuguese, Italian and Japanese, with punctuated, capitalized output. They are for transcribe.cpp.

Rebuildable. Every file is built from IBM's official files by the pinned recipe in conversion/. sh conversion/build.sh WORK_DIR rebuilds all seven and checks them byte for byte against SHA256SUMS.
Checked against IBM's weights. The F32 file holds IBM's checkpoint exactly, and every quantized file is made from it.
Measured. The word error rate of every file on 19,330 English speech clips, and its speed on an Apple M4 Pro, are below.

Voiceour, a local dictation app for macOS, ships the Q8_0 file as its default recognizer.

The transcribe.cpp authors' own handy-computer/granite-speech-5.0-470m-turboctc-gguf holds IBM's earlier English-only release, converted from revision 18ca3c1. transcribe.cpp 0.4.1's converter cannot read the October release without a patch; see How the files were made.

Files

file	size	quantized tensors	use
`granite-speech-5.0-470m-turboctc-Q8_0.gguf`	506 MB	163 Q8_0	the default choice: accuracy indistinguishable from F16 at half the size
`granite-speech-5.0-470m-turboctc-Q6_K.gguf`	392 MB	146 Q6_K, 17 Q8_0	smaller
`granite-speech-5.0-470m-turboctc-Q5_K_M.gguf`	336 MB	130 Q5_K, 33 Q8_0	smaller
`granite-speech-5.0-470m-turboctc-Q4_K_M.gguf`	279 MB	130 Q4_K, 33 Q8_0	smallest
`granite-speech-5.0-470m-turboctc-F16.gguf`	948 MB	163 F16	half precision
`granite-speech-5.0-470m-turboctc-BF16.gguf`	948 MB	131 BF16, 48 F16	IBM's own BF16 weights; transcribe.cpp's reference layout
`granite-speech-5.0-470m-turboctc-F32.gguf`	1.89 GB	none: all 520 tensors F32	IBM's weights exactly; the source of every quantized file

Each file holds 520 tensors. The 163 that the presets quantize are the linear layers and the pointwise convolutions. The other 357 stay F32 in every file: the 16 depthwise convolution kernels, the norms, biases and BatchNorm statistics, and the two frontend tables. BF16 is the exception: it keeps IBM's linear weights in their original BF16 and stores all 48 convolution kernels, depthwise included, in F16.

Usage

hf download vladers/granite-speech-5.0-470m-turboctc-GGUF granite-speech-5.0-470m-turboctc-Q8_0.gguf --local-dir .

git clone https://github.com/handy-computer/transcribe.cpp
cmake -S transcribe.cpp -B build
cmake --build build --target transcribe-cli -j

build/bin/transcribe-cli -m granite-speech-5.0-470m-turboctc-Q8_0.gguf audio.wav

Input is 16 kHz mono WAV (ffmpeg -i in.mp3 -ar 16000 -ac 1 audio.wav). Read Known behaviour before using it on silence or very short clips.

Accuracy

Raw word error rate (%) after the Whisper English text normalizer, before any post-processing, greedy CTC, on Metal, with transcribe.cpp 5700f127: release 0.4.1 plus two commits that touch only Whisper and documentation. Every run had zero errors except the five 40–60 ms clips in Known behaviour, scored as empty.

Confirmation set, 17,858 clips:

file	all (17,858)	AMI (12,393)	FLEURS (719)	LibriSpeech clean (2,210)	LibriSpeech other (2,536)
F32	5.30	8.48	4.53	1.28	2.68
BF16	5.30	8.48	4.50	1.28	2.67
F16	5.30	8.48	4.53	1.28	2.68
Q8_0	5.30	8.48	4.50	1.30	2.67
Q6_K	5.29	8.46	4.54	1.29	2.68
Q5_K_M	5.32	8.52	4.53	1.30	2.68
Q4_K_M	5.39	8.60	4.66	1.33	2.76
reference: Parakeet TDT 0.6B v3, f16	7.21	11.62	5.02	1.98	3.71

Screen set, 1,495 clips:

file	natural speech (1,000)	AMI	FLEURS	LibriSpeech clean	LibriSpeech other	synthetic jargon (456)	technical terms (16)
F32	3.83	7.56	5.12	1.13	2.36	6.56	8.65
BF16	3.82	7.54	5.14	1.11	2.36	6.54	8.65
F16	3.83	7.56	5.12	1.13	2.36	6.56	8.65
Q8_0	3.84	7.50	5.24	1.13	2.36	6.52	8.65
Q6_K	3.75	7.46	4.99	1.11	2.31	6.56	10.58
Q5_K_M	3.89	7.56	5.33	1.13	2.40	6.45	10.58
Q4_K_M	3.95	7.50	5.47	1.26	2.42	6.41	10.58
reference: Parakeet TDT 0.6B v3, f16	4.71	9.31	5.11	1.89	3.57	6.13	6.73

The corpora are AMI IHM meetings, FLEURS en-US, LibriSpeech test-clean and test-other, synthetic technical speech, and 23 non-speech clips. Their selection is described in Voiceour's benchmark record; the confirmation set shares no clip with the screen set. The technical-term set is 16 clips, so one word moves it by about 2 points. On the larger sets, differences under about 0.1 point between precisions run in both directions and are not meaningful. Only English was measured; IBM's model card reports the other languages.

Speed

Apple M4 Pro, macOS 27.0.1, on AC power, transcribe.cpp 5700f127 on Metal with 8 threads. Each file decoded the same 80 clips (822 s of AMI, FLEURS and LibriSpeech audio, 0.3–33 s each) seven times, over three fresh processes with three warm-ups each; every clip gave the same text in all seven passes, for every file.

file	median ms	p95 ms	first result after launch, ms	GPU J per audio minute
F32	40.9	120.2	168	7.15
BF16	39.0	120.7	140	6.61
F16	38.5	114.9	144	6.59
Q8_0	39.4	117.2	136	6.78
Q6_K	41.3	122.3	133	7.26
Q5_K_M	43.1	127.3	134	7.47
Q4_K_M	40.5	120.6	133	7.04
control: 17slever17's Q8_0	39.3	118.5	134	6.78

Milliseconds are model time per clip, from 16 kHz samples to final text, excluding the WAV read. First result is the median of three new processes, each transcribing one 6 s clip; operating system caches were not purged, so it is not a first-install time. Energy is the system GPU rail only. On this machine size buys no speed: against the control, per-clip time is 1% shorter for F16 and 3%, 4% and 8% longer for Q4_K_M, Q6_K and Q5_K_M (geometric mean over the 80 clips). Memory is not compared: the process counters miss most of the weights (F32's peak footprint read 405 MB), so they cannot rank the files.

How the files were made

Source. IBM's six files at revision 1dd23f46, verified against IBM's repository: model.safetensors by the SHA-256 the Hub publishes (5b569c37…7f6fc), the five small files by their git blob ids. Their SHA-256s are in conversion/SOURCE.sha256, which build.sh checks.
Converter. transcribe.cpp release 0.4.1 (4d7c1cce) with two patches:
- granite5_ctc_multilingual.patch by 17slever17 (Apache-2.0, line endings normalized). The October revision ships processor_config.json instead of preprocessor_config.json, and release 0.4.1 stops on it.
- granite5_ctc_deterministic_frontend.patch, written for this repository. See Frontend tables.
Conversion. F32 and BF16 straight from the checkpoint; F16, Q8_0, Q6_K, Q5_K_M and Q4_K_M by transcribe-quantize from the F32 file.

conversion/build.sh does all of it, with the Python environment pinned in conversion/requirements.txt (Python 3.13, torch 2.10.0, CPU wheels).

What was verified

Scripts and their output are in conversion/verification/.

F32 is IBM's checkpoint. 502 of its tensors are exact float32 upcasts of IBM's tensors, and the other 16 are exact concatenations of each block's k_proj and v_proj, which the converter fuses into attn.kv. Only the 16 BatchNorm num_batches_tracked training counters are dropped (verify_source.json).
BF16 keeps IBM's bits. 454 of 518 tensors are bit-identical to IBM's. The rest are the 16 fused attn.kv tensors and 48 convolution kernels stored in F16, because transcribe.cpp has no BF16 convolution kernel. That is also why quantizing from BF16 would differ, and every quantized file here comes from F32.
The weights match an independent build. The Q8_0 file has the same 64 metadata entries and the same 518 weight tensors, byte for byte, as 17slever17's independent conversion. All 163 quantized tensors are identical. Only the two frontend tables differ.
The build repeats. A from-scratch run of build.sh re-cloned transcribe.cpp, re-downloaded IBM's files, recreated the Python environment and recompiled the quantizer. It reproduced all seven files byte for byte.

Frontend tables

Besides the weights, the converter bakes two constant tables into the GGUF: the 80×257 mel filterbank and the 400-sample Hann window. Upstream computes them with torch in float32, so their last bits depend on the machine that runs the converter. A stock conversion here and 17slever17's, from the same checkpoint, disagree in 7 window values (by up to 6e-8) and in the sign of one zero.

The deterministic-frontend patch evaluates the same formulas in float64 and rounds once to float32. It also pins the mel end points to exactly 0 Hz and 8 kHz and normalizes -0.0 to +0.0. verify_frontend.json shows:

every one of the 20,960 values is the correctly rounded result of the exact formula, checked against 60-digit arithmetic;
shifting every non-exact log10, pow and cos result by up to ±4 ulp, as another maths library might, changed no byte in 2,002 runs. The closest value is 32,002 float64 ulps from a rounding boundary.

The tables differ from torch's float32 ones by at most 5.3e-6 (filterbank) and 2.4e-7 (window). Against 17slever17's otherwise identical Q8_0, that changed 2 of the 1,495 screen transcripts, neither in its words after normalization, and 23 of the 17,858 confirmation transcripts, 11 of them in their words. Confirmation word edits went from 9,468 to 9,467.

Known behaviour

In transcribe.cpp 0.4.1:

Non-speech returns a control token as text. On silence, noise and music the model emits its <|nospeech|> control token, and transcribe.cpp returns it as the transcript. That happened for 22 of 23 non-speech clips, sometimes followed by .. 8 s of silence returned Il. instead. Drop special tokens (ids 0–254) from the text, and treat output that is only <|nospeech|> as no speech.
Clips shorter than 1,120 samples (70 ms) fail with run: gguf load error. The model loaded; the input is too short for one encoder frame. Right-pad shorter audio with zeros to 1,120 samples.
Verbatim speech loses disfluencies, and sometimes words. The model tends to drop the repetitions and fillers that AMI's verbatim references keep, and occasionally content: I don't know. became I'.. Voiceour's paired evaluation against Parakeet counts these deletions.

License

Apache-2.0, the licence of IBM's model (LICENSE). NOTICE states what was changed and credits the tools. The weights are IBM's work; this repository is not an official IBM release.

vladers/granite-speech-5.0-470m-turboctc-GGUF