This repository provides an Apple-Silicon MLX 4-bit conversion of khleeloo/whisper-large-v3-cantonese. The original Cantonese fine-tuning was performed by the upstream model author. This repository contributes MLX conversion, 4-bit quantisation, validation, reproducibility information, and Apple-Silicon benchmarking.
At the time of our search on 2026-09-26, we did not identify another publicly downloadable MLX 4-bit conversion of this specific upstream Cantonese checkpoint.
whisper-large-v3-cantonese is a full fine-tune of openai/whisper-large-v3 for Cantonese (yue) automatic speech recognition, trained by khleeloo (Rita Frieske) on Common Voice 17 yue. This repository converts that checkpoint to Apple's MLX array framework and quantises it to 4-bit (group size 64) so it runs efficiently on Apple Silicon via mlx-whisper.
khleeloo/whisper-large-v3-cantonese, revision f48a890f78c7b6acf723f25d8c81e232ac7469ca.openai/whisper-large-v3 (Apache-2.0), full fine-tune (not LoRA/adapter).yue.openai/whisper-large-v3 dimensions. This is a full-weight checkpoint, not a LoRA/adapter; the upstream model card reports 12 decoder layers frozen during fine-tuning (10 epochs, learning rate 1e-7).generation_config.json on the upstream repo pins language = <|yue|>.whisper-large-v3.Cantonese (Hong Kong / yue) speech-to-text on Apple Silicon Macs, via mlx-whisper. Suitable for local, offline transcription workflows that need Traditional-script Cantonese output. Not evaluated for languages other than Cantonese, and not intended as a general-purpose multilingual Whisper replacement.
mlx (tested with mlx==0.32.2) and mlx-whisper (tested with mlx-whisper==0.4.3).pip install mlx-whisper
import mlx_whisper
result = mlx_whisper.transcribe(
"audio.wav",
path_or_hf_repo="WinsonLi/whisper-large-v3-cantonese-mlx-4bit",
language="yue",
)
print(result["text"])
Always pass language="yue" explicitly. In our own conversion, decoding the same 4-bit/fp16 weights with language="zh" instead of language="yue" produced Mandarin wording on our test takes (e.g. 這個 / 我們 / 不要 in place of the Cantonese 呢個 / 我哋 / 唔好), even though the underlying weights are unchanged. This is an implementation observation from our own smoke test, not a claim about how zh behaves in every context. The yue language token pin is load-bearing for Cantonese output.
mlx.nn.quantize, affine quantisation.weights.safetensors header). Layer norms and the two encoder convolution layers (encoder.conv1, encoder.conv2) remain float16.n_mels=128, n_audio_layer=32, n_text_layer=32, n_vocab=51866, n_audio_state=1280, n_text_state=1280, n_audio_head=20, n_text_head=20, n_audio_ctx=1500, n_text_ctx=448.weights.safetensors: 877,695,978 bytes (≈837 MiB / 878 MB).weights.safetensors): 7ea0e37f88ac3a4cdb00435a3418498ff707ba7b1cc45d1feb13bd82ec471374.Full checksums for every file in this repository are in SHA256SUMS.
Conversion is deterministic: repeating it from the same source checkpoint produces byte-identical weights.safetensors (verified by re-running the conversion and comparing hashes).
# 1. Download the pinned upstream revision (this is a gated-with-auto-approval
# repository on Hugging Face; you must accept the terms once via the HF UI
# or CLI before your token can download it).
hf download khleeloo/whisper-large-v3-cantonese \
--revision f48a890f78c7b6acf723f25d8c81e232ac7469ca \
--local-dir <local-hf-checkpoint-dir> \
--exclude "runs/*" --exclude "training_args.bin"
# 2. Convert to MLX and quantise to 4-bit.
# convert.py in this repository is ml-explore/mlx-examples' whisper/convert.py
# (commit 796f5b53cab69a3d48a44233ce21aae889e94a08, MIT licence), patched to
# merge sharded safetensors (khleeloo ships model-00001-of-00002.safetensors +
# model-00002-of-00002.safetensors + an index file, whereas upstream convert.py
# only reads a single model.safetensors) and to name the output weights.safetensors.
python convert.py \
--torch-name-or-path <local-hf-checkpoint-dir> \
--mlx-path <output-dir> \
--dtype float16 -q --q-bits 4 --q-group-size 64
Runtime used for conversion and validation: Python 3.13.7, mlx 0.32.2, mlx-whisper 0.4.3.
mx.get_peak_memory()) during load plus all 23 decodes, not RSS.initial_prompt containing proper nouns and technical terms (Inflow, JCode, WezTerm, tmux, MCP, GitHub, …). SenseVoice and Qwen3 ran with no prompt and no hotwords. Proper-noun and technical-term gaps between the Whisper rows and those two rows are therefore prompt-assisted vs. unassisted, not a pure model-vs-model comparison.| Variant | CER | CER HK | HK markers | Proper nouns | Code-switch CER (MX) | Punct./cmd CER (n=7) | Script T/S | Median | P95 | Peak RAM | Model size | Note |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SenseVoice | 0.201 | 0.172 | 0.68 | 0.38 | 0.238 | 0.292 | 0/21 | 0.03 s | 0.04 s | n/m (RSS ≥ 1.0 GB, lower bound) | 936 MB | Unprompted (see prompt-asymmetry note); Simplified output |
| whisper-large-v3-turbo | 0.285 | 0.268 | 0.04 | 0.59 | 0.306 | 0.307 | 20/2 | 0.37 s | 0.39 s | 2365 MB | 1.6 GB | Prompted; Mandarin wording (HK markers 0.04) |
| Qwen3-ASR-1.7B 4-bit | 0.163 | 0.180 | 0.75 | 0.42 | 0.140 | 0.242 | 4/17 | 0.25 s | 0.36 s | n/m (RSS ≥ 1.75 GB, lower bound) | 1.6 GB | Unprompted (see prompt-asymmetry note); mostly Simplified |
| whisper-large-v2-cantonese (doggy8088) 4-bit | 0.148 | 0.112 | 0.96 | 0.68 | 0.195 | 0.213 | 22/0 | 0.65 s | 0.78 s | 1671 MB | 877 MB | Prompted |
| This model (v3-Cantonese 4-bit) | 0.132 | 0.093 | 1.00 | 0.84 | 0.183 | 0.176 | 22/0 | 0.64 s | 1.03 s (per-take median: 0.69 s) | 1675 MB | 878 MB | Prompted |
| v3-Cantonese fp16 (reference, not released) | 0.130 | 0.088 | 1.00 | 0.84 | 0.183 | 0.173 | 22/0 | 0.70 s | 1.17 s | 3766 MB | 3.1 GB | Prompted; +2.1 GB RAM for Δ CER 0.002 vs. 4-bit |
terms_kept case-sensitivity bug that under-scores exact matches with capitalised terms; a markers_kept check that penalises Simplified-script output; a diluted/substring-matching proper_nouns_kept average; references that spell out numbers so that any engine normalising to digits (e.g. "500ms") is penalised; and an MLX-memory undercounting bug in peak_rss_mb (peak MLX memory was measured directly with mx.get_peak_memory() instead, as shown above).Observed on our private test takes; not exhaustive:
tmux is misrecognised as "tmas" (present in both 4-bit and fp16 — a model behaviour, not a quantisation artefact).None of these are fixed by this conversion; they reflect the underlying fine-tune's behaviour (or, for the two quantisation-specific cases, the 4-bit quantisation) and would need a deterministic protected-term/post-processing layer to correct.
weights.safetensors, config.json): Apache-2.0, inherited from openai/whisper-large-v3 and from khleeloo/whisper-large-v3-cantonese, both licensed Apache-2.0. Full licence text is in LICENSE.convert.py: derived from ml-explore/mlx-examples whisper/convert.py (commit 796f5b53cab69a3d48a44233ce21aae889e94a08), MIT licence, Copyright © 2023-2024 Apple Inc. Patched only to merge sharded safetensors and to write weights.safetensors; see the file header for the exact patch description. The MIT copyright and permission notice is included in full in LICENSE-convert.py-MIT.yue) is not redistributed here; Mozilla Common Voice is released under CC0-1.0 (public domain dedication) per Mozilla's own project terms.yue training data used in the upstream fine-tune.whisper-large-v2-cantonese-mlx-4bit as a prior MLX Cantonese Whisper reference point used in our own benchmarking.If you use this model, please cite the upstream fine-tune:
@misc {rita_frieske_2025,
author = { {Rita Frieske} },
title = { whisper-large-v3-cantonese },
year = 2025,
url = { https://huggingface.co/khleeloo/whisper-large-v3-cantonese },
doi = { 10.57967/hf/4393 },
publisher = { Hugging Face }
}
The original measurements above are preserved as historical results. A new matched test on the same 23 completed private Engineer Mode takes compared this exact 4-bit checkpoint with direct-from-original 3-bit and 2-bit conversions. The corrected conventional normalized corpus CER was 14.92% (4-bit), 24.50% (3-bit), and 100% (2-bit g64) using the same static prompt. Without a prompt it was 16.37%, 18.26%, and 100%, respectively. The old 0.132 score is a nonstandard per-take symmetric mean and should not be compared directly with conventional corpus CER.
The 2-bit g64 artifact is 43.7% smaller but emitted only punctuation/whitespace on every primary take. The finer 2-bit g32 diagnostic also failed. Keep 4-bit as the recommended baseline. The 3-bit result is prompt-sensitive and is not evidence of accuracy equivalence.
Full method, corrected metrics, actual disk/memory/timing measurements and failed 2-bit research artifact. No original 4-bit weights or earlier benchmark results were replaced. Private recordings and transcripts remain private.
khleeloo/whisper-large-v3-cantonese