Granite Speech 5.0 470M TurboCTC — MLX 8-bit

This is an unofficial MLX conversion of IBM's Granite Speech 5.0 470M TurboCTC, with 8-bit affine weight quantization, in the layout used by the Granite-MLX Swift library, for speech recognition on Apple Silicon Macs.

Source: ibm-granite/granite-speech-5.0-470m-turboctc at revision 91dcb0605daf65f88dbb04fef015c423241d941d. This is the multilingual release of 21 October 2026: English, French, German, Spanish, Portuguese, Italian and Japanese, with native punctuation and capitalization.

All model credit belongs to IBM Granite. This repository changes the weight layout (MLX safetensors as read by Granite-MLX), the weight precision (8-bit affine), config.json (adds the quantization block), one tokenizer setting, and adds change notices. IBM has not reviewed or endorsed it.

The original model

IBM describes the model as 16 conformer blocks trained with CTC, with a 16,384-token multilingual BPE head. It uses 8× temporal subsampling (100 Hz to 12.5 Hz), block attention over 128 frames, and self-conditioned CTC from the middle layer: 470M parameters in total. For training data, intended use, benchmarks and the technical report, see the IBM model card at the pinned revision.

Licence and permission to republish

IBM releases this model under the Apache License 2.0. The pinned source revision declares license: apache-2.0 in its card metadata, and its README states:

License: Apache 2.0

(source, revision 91dcb06)

Apache-2.0 section 2 grants the right to "reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form". Section 4 requires that redistribution include a copy of the licence and a notice of changes. Accordingly:

• LICENSE is the verbatim Apache-2.0 text from apache.org (SHA-256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30). The source repository ships no separate LICENSE or NOTICE file at this revision.
• MODIFICATIONS.md states what was changed, and each changed file carries its own notice: the safetensors header metadata has license: Apache-2.0 and a modification_notice field, tokenizer_config.json has a _modification_notice key, and config.json has a _modification_notice key. Unchanged source files are byte-identical.

Do not confuse this model with ibm-granite/granite-speech-5.0-470m-turboctc-nc, a different, English-only model under CC-BY-NC-SA-4.0. That model was not used here.

What this conversion is

How it was produced. convert_granite.py from Granite-MLX at commit 7ad6b19c698aac5aa8328c911e4a6eead456c5a1, run unmodified: --precision fp16 --quantization-bits 8 --group-size 128 --source-model-id ibm-granite/granite-speech-5.0-470m-turboctc --source-revision 91dcb0605daf65f88dbb04fef015c423241d941d. It keeps the 534 source tensors (stored as 892 arrays, because each of the 179 quantized matrices adds scale and bias arrays), drops 16 BatchNorm num_batches_tracked counters (training-only), and transposes the 16 depthwise Conv1d weights from [O,I,K] to MLX's [O,K,I]. The license and modification_notice entries were then added to the safetensors header metadata, with the tensor bytes unchanged (hash-verified). Versions: Python 3.12, mlx 0.31.2, torch 2.13.0, transformers 5.16.1 (reference inference). Run on macOS 27.0.1, Apple M2 Max, on 2026-11-12/13 (America/Los_Angeles). Full provenance is in CONVERSION.json.

Files

Every file's SHA-256 is listed in SHA256SUMS; check with shasum -a 256 -c SHA256SUMS.

Tokenizer deviation. tokenizer_config.json adds "clean_up_tokenization_spaces": false and a _modification_notice string (the Apache-2.0 §4(b) change notice; Python transformers and swift-transformers both load the file with it, tested); nothing else differs from the source file (source SHA-256 2f61ece3468bf7ca7a1b6dd3fd5b809814315bea9c298feea035fda4931b9f02). swift-transformers 1.3.3 turns that cleanup on when the key is absent, and it strips the space before ? ! : ;, which corrupts French typography. Python transformers already leaves it off for this tokenizer, so Python output is unchanged.

How to use it

Swift / command line: Granite-MLX

Tested with the granite-mlx CLI 0.1.1, built from Granite-MLX 7ad6b19 with swift build -c release --product granite-mlx (Swift 6.4), on a local copy of this repository:

granite-mlx transcribe audio.wav --model /path/to/granite-speech-5.0-470m-turboctc-mlx-8bit --backend mlx \
  --no-punctuate --output-format txt --output-dir out

--no-punctuate disables Granite-MLX's separate punctuation model; this Granite release punctuates natively. Use --output-format txt: compatibility was validated for raw text only. The CLI's default subtitle output (SRT) and other word-timed formats run words together with this tokenizer (see Limitations). Loading by Hugging Face repository id was not tested.

No Python loader for this layout was tested.

Validation

Method. The reference is IBM's model in transformers 5.16.1, fp32 on CPU, variable length with an attention mask, with greedy CTC (official inference). This package was run through the Granite-MLX Swift CLI, which uses its own Swift feature frontend.

The fixed corpus has 141 unique clips:

• 73 from LibriSpeech (hf-internal-testing/librispeech_asr_dummy);
• 60 from the FLEURS test split (the first 10 rows in file order for each of fr, de, es, pt, it and ja);
• 4 English local clips: the public-domain JFK inaugural excerpt (11 s) and 3 private recordings (30 s, 1 min, 5 min) that are not published;
• 4 non-speech clips: digital silence of 5 s and 30 s, and seeded white and pink noise.

Long-audio behaviour was tested separately on 4 constructed clips of 286.5–328 s. Text is compared after NFKC normalization, lower-casing and removing punctuation; WER is used for European languages, and CER with whitespace removed for Japanese. All rates are corpus-level per set. With 10 clips per FLEURS language these are small samples: they show conversion fidelity, not benchmark accuracy.

Results for this 8-bit package, in percent:

Against a chunked official reference that uses the same 286.72 s policy as Granite-MLX, the long clips differ by 1 normalized word in 3,672.

Size and speed. model.safetensors is 488,666,432 bytes. Measured with the Granite-MLX CLI, 5 interleaved rounds: median 0.10 s on an 11 s clip, 0.16 s on a 1-minute clip and 0.72 s on a 328 s clip, peak RSS 0.65 GB. These timings were measured on a busy machine (1-minute load average about 6–7) and are indicative only.

Limitations

• Close to IBM's own inference, not bit-identical. The MLX path runs variable-length input, as IBM's code does, and no final-period loss was observed on the 141-clip corpus (the fixed-shape Core ML packages drop it on 119 of those clips). The remaining differences come from fp16 and 8-bit arithmetic and the Swift frontend (table above).
• 8-bit cost. Compared with our MLX fp16 package, the worst case is 0.43% (it), with pt at 0.39% and LibriSpeech at 0.09%.
• Malformed packages are not always rejected cleanly. If config.json loses its quantization block, the Granite-MLX 7ad6b19 CLI crashes (SIGTRAP, addmm shape mismatch) instead of reporting an error. Verify SHA256SUMS before use.
• Silence and noise. The model emits a literal <|nospeech|> token, which decode returns as text unless you filter it.
• Long audio. Granite-MLX chunks audio longer than 286.72 s (when run with --audio-chunk-duration 286.72 --audio-chunk-context 20.48, as tested) into windows with 20.48 s of context on each side, trimming context in frame space before CTC collapse. Results near chunk joins depend on that policy (see Validation).
• Swift tokenizer. Granite-MLX 7ad6b19 loads this tokenizer only by forcing the GPT-2 tokenizer class (the source class is ParakeetTokenizer, which swift-transformers 1.3.3 rejects). Decoding is correct for all seven languages in our tests when the shipped tokenizer_config.json is used.
• Subtitle and word-timing output in Granite-MLX 7ad6b19. The CLI's default SRT output (and other word-timed formats) joins words without spaces, for example Andso,myfellowAmericans,..., because the timing path decodes one token at a time and this tokenizer's decoder strips each token's leading space. Raw text (--output-format txt) is correct. A fix belongs in Granite-MLX.
• Native punctuation. The model punctuates and capitalizes by itself. If your application also runs a separate punctuation or formatting model, expect overlap.
• No Neural Engine placement, quiet-host latency, streaming or iOS measurements were made.