Granite Speech 5.0 470M TurboCTC — MLX 4-bit

This is an unofficial MLX conversion of IBM's Granite Speech 5.0 470M TurboCTC, with 4-bit affine weight quantization, in the layout used by the Granite-MLX Swift library, for speech recognition on Apple Silicon Macs.

Source: ibm-granite/granite-speech-5.0-470m-turboctc at revision 91dcb0605daf65f88dbb04fef015c423241d941d. This is the multilingual release of 21 October 2026: English, French, German, Spanish, Portuguese, Italian and Japanese, with native punctuation and capitalization.

All model credit belongs to IBM Granite. This repository changes the weight layout (MLX safetensors as read by Granite-MLX), the weight precision (4-bit affine), config.json (adds the quantization block), one tokenizer setting, and adds change notices. IBM has not reviewed or endorsed it.

Read this first: 4-bit costs accuracy

Prefer the 8-bit package (swaynehales/granite-speech-5.0-470m-turboctc-mlx-8bit) unless size or memory matters more than accuracy. This 4-bit package is a disclosed size-over-accuracy option, with no accuracy gate claimed. Its cost is several times the 8-bit cost, and it is largest on Portuguese (3.88% against fp16). It is no faster than 8-bit in our timing.

Word or character difference in percent (normalized; lower is closer), measured through the Granite-MLX Swift CLI:

FLEURS sets have only 10 clips each.

The original model

IBM describes the model as 16 conformer blocks trained with CTC, with a 16,384-token multilingual BPE head. It uses 8× temporal subsampling (100 Hz to 12.5 Hz), block attention over 128 frames, and self-conditioned CTC from the middle layer: 470M parameters in total. For training data, intended use, benchmarks and the technical report, see the IBM model card at the pinned revision.

Licence and permission to republish

IBM releases this model under the Apache License 2.0. The pinned source revision declares license: apache-2.0 in its card metadata, and its README states:

License: Apache 2.0

(source, revision 91dcb06)

Apache-2.0 section 2 grants the right to "reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form". Section 4 requires that redistribution include a copy of the licence and a notice of changes. Accordingly:

• LICENSE is the verbatim Apache-2.0 text from apache.org (SHA-256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30). The source repository ships no separate LICENSE or NOTICE file at this revision.
• MODIFICATIONS.md states what was changed, and each changed file carries its own notice: the safetensors header metadata has license: Apache-2.0 and a modification_notice field, tokenizer_config.json has a _modification_notice key, and config.json has a _modification_notice key. Unchanged source files are byte-identical.

Do not confuse this model with ibm-granite/granite-speech-5.0-470m-turboctc-nc, a different, English-only model under CC-BY-NC-SA-4.0. That model was not used here.

What this conversion is

How it was produced. convert_granite.py from Granite-MLX at commit 7ad6b19c698aac5aa8328c911e4a6eead456c5a1, run unmodified: --precision fp16 --quantization-bits 4 --group-size 64 --source-model-id ibm-granite/granite-speech-5.0-470m-turboctc --source-revision 91dcb0605daf65f88dbb04fef015c423241d941d. It keeps the 534 source tensors (stored as 892 arrays, because each of the 179 quantized matrices adds scale and bias arrays), drops 16 BatchNorm num_batches_tracked counters (training-only), and transposes the 16 depthwise Conv1d weights from [O,I,K] to MLX's [O,K,I]. Group size 64 matches the existing community recipe iky1e/granite-speech-5.0-470m-turboctc-mlx-q4 (older English-only checkpoint). The license and modification_notice entries were then added to the safetensors header metadata, with the tensor bytes unchanged (hash-verified). Versions: Python 3.12, mlx 0.31.2, torch 2.13.0, transformers 5.16.1 (reference inference). Run on macOS 27.0.1, Apple M2 Max, on 2026-11-12/13 (America/Los_Angeles). Full provenance is in CONVERSION.json.

Files

Every file's SHA-256 is listed in SHA256SUMS; check with shasum -a 256 -c SHA256SUMS.

Tokenizer deviation. tokenizer_config.json adds "clean_up_tokenization_spaces": false and a _modification_notice string (the Apache-2.0 §4(b) change notice; Python transformers and swift-transformers both load the file with it, tested); nothing else differs from the source file (source SHA-256 2f61ece3468bf7ca7a1b6dd3fd5b809814315bea9c298feea035fda4931b9f02). swift-transformers 1.3.3 turns that cleanup on when the key is absent, and it strips the space before ? ! : ;, which corrupts French typography. Python transformers already leaves it off for this tokenizer, so Python output is unchanged.

How to use it

Swift / command line: Granite-MLX

Tested with the granite-mlx CLI 0.1.1, built from Granite-MLX 7ad6b19 with swift build -c release --product granite-mlx (Swift 6.4), on a local copy of this repository:

granite-mlx transcribe audio.wav --model /path/to/granite-speech-5.0-470m-turboctc-mlx-4bit --backend mlx \
  --no-punctuate --output-format txt --output-dir out

--no-punctuate disables Granite-MLX's separate punctuation model; this Granite release punctuates natively. Use --output-format txt: compatibility was validated for raw text only. The CLI's default subtitle output (SRT) and other word-timed formats run words together with this tokenizer (see Limitations). Loading by Hugging Face repository id was not tested.

No Python loader for this layout was tested.

Validation

Method. The reference is IBM's model in transformers 5.16.1, fp32 on CPU, with greedy CTC (official variable-length inference). This package and our fp16 and 8-bit packages were run through the Granite-MLX Swift CLI on the same corpus.

The fixed corpus has 141 unique clips:

• 73 from LibriSpeech (hf-internal-testing/librispeech_asr_dummy);
• 60 from the FLEURS test split (the first 10 rows in file order for each of fr, de, es, pt, it and ja);
• 4 English local clips: the public-domain JFK inaugural excerpt (11 s) and 3 private recordings (30 s, 1 min, 5 min) that are not published;
• 4 non-speech clips: digital silence of 5 s and 30 s, and seeded white and pink noise.

Long-audio behaviour was tested separately on 4 constructed clips of 286.5–328 s. Text is compared after NFKC normalization, lower-casing and removing punctuation; WER is used for European languages, and CER with whitespace removed for Japanese. All rates are corpus-level per set. With 10 clips per FLEURS language these are small samples: they show conversion fidelity, not benchmark accuracy.

Further results for this 4-bit package, in percent (the cost table is at the top):

Size and speed. model.safetensors is 267,317,480 bytes (8-bit: 488,666,432). Measured with the Granite-MLX CLI, 5 rounds interleaved with the 8-bit package: median 0.087 s on an 11 s clip and 0.746 s on a 328 s clip (8-bit: 0.091 s and 0.745 s); peak RSS 0.43 GB (8-bit 0.65 GB). These timings were measured on a busy machine (1-minute load average about 6–8) and are indicative only.

Limitations

• 4-bit cost. See the table at the top. This package is noticeably less accurate than the 8-bit and fp16 packages, most of all on Portuguese.
• Final punctuation. The MLX path runs variable-length input, as IBM's code does, so it avoids the fixed-shape padding mechanism that drops the final period in the Core ML packages. 4-bit weights still change punctuation, though: compared with official inference, this package dropped the final sentence mark (. or 。) on 4 of the 141 clips (1 Spanish, 3 Japanese); the fp16 and 8-bit MLX packages dropped none. Punctuation is not guaranteed identical.
• Malformed packages are not always rejected cleanly. A missing tokenizer.json is rejected; with the 8-bit package, a config.json without its quantization block crashed the Granite-MLX 7ad6b19 CLI instead of reporting an error. Verify SHA256SUMS before use.
• Silence and noise. The model emits a literal <|nospeech|> token, which decode returns as text unless you filter it.
• Long audio. Granite-MLX chunks audio longer than 286.72 s (when run with --audio-chunk-duration 286.72 --audio-chunk-context 20.48, as tested) into windows with 20.48 s of context on each side, trimming context in frame space before CTC collapse. Results near chunk joins depend on that policy (see Validation).
• Swift tokenizer. Granite-MLX 7ad6b19 loads this tokenizer only by forcing the GPT-2 tokenizer class (the source class is ParakeetTokenizer, which swift-transformers 1.3.3 rejects). Decoding is correct for all seven languages in our tests when the shipped tokenizer_config.json is used.
• Subtitle and word-timing output in Granite-MLX 7ad6b19. The CLI's default SRT output (and other word-timed formats) joins words without spaces, for example Andso,myfellowAmericans,..., because the timing path decodes one token at a time and this tokenizer's decoder strips each token's leading space. Raw text (--output-format txt) is correct. A fix belongs in Granite-MLX.
• Native punctuation. The model punctuates and capitalizes by itself. If your application also runs a separate punctuation or formatting model, expect overlap.
• No Neural Engine placement, quiet-host latency, streaming or iOS measurements were made.