Qwen3-ASR 0.6B for Core ML

This is Qwen3-ASR 0.6B converted to Core ML so it runs locally on Apple silicon Macs. It was made for Speakey, a dictation and voice-command app for the Mac, and Speakey downloads it on its own. The weights are Qwen's and were not retrained.

The audio encoder runs on the Apple Neural Engine in fp16. The text decoder runs on the GPU with int8 weights and keeps its KV cache in Core ML state, which needs macOS 15 or later.

There is also a 1.7B version, more accurate, about twice the size and decoding time.

Files

File	Size	What it is
`encoder.mlmodelc`	356 MB	Audio encoder, fp16, Neural Engine
`decoder.mlmodelc`	571 MB	Text decoder, int8 weights, GPU, functions `main` and `step`
`embeddings.i8`	148 MB	Token embeddings, int8, 151936 × 1024
`embeddings.scale`	< 1 MB	One float32 scale per embedding row
`vocab.json`	2.6 MB	Qwen's byte-level BPE vocabulary
`mel.bin`	100 KB	128 × 201 float32 mel filterbank (Whisper, Slaney)
`speakey.json`	4 KB	Shapes, special token ids, the prompt around the audio, and the `language X<asr_text>` ids for each language
`LICENSE`, `NOTICE`		Apache 2.0, and what was changed

Total download: about 1.1 GB.

Architecture


Audio encoder	3 conv2d layers per 100-frame chunk, then 18 transformer layers of width 896; attention stays inside 8-second windows
Encoder output	104 tokens of width 1024 per 8-second window
Text decoder	Qwen3, 28 layers, hidden size 1024, MLP 3072, 16 query heads and 8 KV heads of 128
KV cache	1024 rows, held as 56 Core ML state tensors (`k0`…`k27`, `v0`…`v27`)
Vocabulary	151,936 tokens

Download

hf download raymondchins/qwen3-asr-0.6b-coreml --local-dir qwen3-asr-0.6b-coreml

Running it outside Speakey

The folder holds compiled models (.mlmodelc), so load them with MLModel(contentsOf:). Speakey's runtime does the following, and any other host needs the same steps.

Audio: 16 kHz mono, turned into a 128-bin log-mel spectrogram the way Whisper does it (400-sample FFT, hop 160, Hann window, mel.bin as the filterbank, log10, floor at max − 8, then (x + 4) / 4). There is no padding to 30 seconds.
Encoder: split the mel into 800-frame windows (8 s) and pad the last one with zeros. Each call takes mel (1, 128, 800) and an additive key mask mask (1, 1, 1, 104), and returns audio (1, 104, 1024). A full 100-frame chunk gives 13 tokens and a partial chunk of n frames gives ⌈n / 8⌉; mask the rest with −1e4. Because the original encoder never attends across windows, this is exact for any length.
Prompt: prefix from speakey.json, then the audio tokens, then suffix. To force a language, append that language's ids. Without them the model writes language X<asr_text> before the text. Look up text token embeddings in embeddings.i8 and multiply each row by its scale.
Decoder: main takes any length for prefill and step takes one token. Both share one MLState. Inputs are embeds (1, S, 1024), place (1, 1, 1024, S), a one-hot that writes token s into cache row r, cos and sin (1, S, 128) for RoPE with θ = 1,000,000, mask (1, 1, S, 1024) additive over cache rows, and sel (1, T, S), which picks the positions whose next token you want.
Output: vals and idx, each (1, T, 16). The LM head is split into 16 parts of 9,496 tokens, so the token is idx[part] + part × 9496 for the part with the largest value. The offset is added on the host because the GPU computed it in 16-bit integers and overflowed. Stop at token 151643 or 151645.

Checking several draft tokens in one main call costs about the same as decoding one, which is how Speakey confirms its live-preview text when a take ends.

Performance

Measured on an M5 Max (macOS 27), twelve Indonesian/English dictation takes of 5.4 to 14.5 seconds, warm:

	p50	p90
Final take, plain greedy decoding	197 ms	286 ms
Final take, checking a live-preview draft	35 ms	64 ms
One decoder step (one token)	4.9 ms
One encoder window (8 s of audio)	9 ms

The same takes through mlx-qwen3-asr (bf16, GPU) took 155 / 229 ms, and Whisper Large v3 Turbo through WhisperKit 529 / 734 ms. Memory footprint while in use: about 370 MB.

Conversion accuracy

Against mlx-qwen3-asr with the original bf16 weights, about 3% of words differed across the twelve takes. Most of that comes from running the encoder in fp16 on the Neural Engine (for example "akurat" where the reference wrote "akhirat"); the int8 decoder gave the same words as fp16.

Limitations

macOS 15 or later on Apple silicon. Core ML state does not exist on earlier systems.
Indonesian and some other languages often come back lowercase and without punctuation. The original model writes them that way too; it is not caused by the conversion.
Neither the KV cache nor the prompt can grow past 1024 rows, so Speakey decodes takes longer than about 40 seconds in parts.
The decoder stays at int8. On the 0.6B model, int4 weights changed 12% of words and were no faster, because each step is limited by dispatch overhead and not by weight bandwidth.

License

Apache License 2.0, the same as the original model. See LICENSE and NOTICE. Qwen3-ASR 0.6B is Copyright (c) Alibaba Cloud (Qwen team). Converted and modified by Raymond Chin for Speakey.

Citation

If you use this model, please cite the original work:

@article{qwen3asr,
  title   = {Qwen3-ASR Technical Report},
  author  = {Qwen Team},
  journal = {arXiv preprint arXiv:2601.21337},
  year    = {2026}
}

raymondchins/qwen3-asr-0.6b-coreml