Qwen3-ASR 0.6B for Core ML

This is Qwen3-ASR 0.6B converted to Core ML so it runs locally on Apple silicon Macs. It was made for Speakey, a dictation and voice-command app for the Mac, and Speakey downloads it on its own. The weights are Qwen's and were not retrained.

The audio encoder runs on the Apple Neural Engine in fp16. The text decoder runs on the GPU with int8 weights and keeps its KV cache in Core ML state, which needs macOS 15 or later.

There is also a 1.7B version, more accurate, about twice the size and decoding time.

Files

Total download: about 1.1 GB.

Architecture

Download

hf download raymondchins/qwen3-asr-0.6b-coreml --local-dir qwen3-asr-0.6b-coreml

Running it outside Speakey

The folder holds compiled models (.mlmodelc), so load them with MLModel(contentsOf:). Speakey's runtime does the following, and any other host needs the same steps.

1. Audio: 16 kHz mono, turned into a 128-bin log-mel spectrogram the way Whisper does it (400-sample FFT, hop 160, Hann window, mel.bin as the filterbank, log10, floor at max − 8, then (x + 4) / 4). There is no padding to 30 seconds.
2. Encoder: split the mel into 800-frame windows (8 s) and pad the last one with zeros. Each call takes mel (1, 128, 800) and an additive key mask mask (1, 1, 1, 104), and returns audio (1, 104, 1024). A full 100-frame chunk gives 13 tokens and a partial chunk of n frames gives ⌈n / 8⌉; mask the rest with −1e4. Because the original encoder never attends across windows, this is exact for any length.
3. Prompt: prefix from speakey.json, then the audio tokens, then suffix. To force a language, append that language's ids. Without them the model writes language X<asr_text> before the text. Look up text token embeddings in embeddings.i8 and multiply each row by its scale.
4. Decoder: main takes any length for prefill and step takes one token. Both share one MLState. Inputs are embeds (1, S, 1024), place (1, 1, 1024, S), a one-hot that writes token s into cache row r, cos and sin (1, S, 128) for RoPE with θ = 1,000,000, mask (1, 1, S, 1024) additive over cache rows, and sel (1, T, S), which picks the positions whose next token you want.
5. Output: vals and idx, each (1, T, 16). The LM head is split into 16 parts of 9,496 tokens, so the token is idx[part] + part × 9496 for the part with the largest value. The offset is added on the host because the GPU computed it in 16-bit integers and overflowed. Stop at token 151643 or 151645.

Checking several draft tokens in one main call costs about the same as decoding one, which is how Speakey confirms its live-preview text when a take ends.

Performance

Measured on an M5 Max (macOS 27), twelve Indonesian/English dictation takes of 5.4 to 14.5 seconds, warm:

The same takes through mlx-qwen3-asr (bf16, GPU) took 155 / 229 ms, and Whisper Large v3 Turbo through WhisperKit 529 / 734 ms. Memory footprint while in use: about 370 MB.

Conversion accuracy

Against mlx-qwen3-asr with the original bf16 weights, about 3% of words differed across the twelve takes. Most of that comes from running the encoder in fp16 on the Neural Engine (for example "akurat" where the reference wrote "akhirat"); the int8 decoder gave the same words as fp16.

Limitations

• macOS 15 or later on Apple silicon. Core ML state does not exist on earlier systems.
• Indonesian and some other languages often come back lowercase and without punctuation. The original model writes them that way too; it is not caused by the conversion.
• Neither the KV cache nor the prompt can grow past 1024 rows, so Speakey decodes takes longer than about 40 seconds in parts.
• The decoder stays at int8. On the 0.6B model, int4 weights changed 12% of words and were no faster, because each step is limited by dispatch overhead and not by weight bandwidth.

License

Apache License 2.0, the same as the original model. See LICENSE and NOTICE. Qwen3-ASR 0.6B is Copyright (c) Alibaba Cloud (Qwen team). Converted and modified by Raymond Chin for Speakey.

Citation

If you use this model, please cite the original work:

@article{qwen3asr,
  title   = {Qwen3-ASR Technical Report},
  author  = {Qwen Team},
  journal = {arXiv preprint arXiv:2601.21337},
  year    = {2026}
}