Qwen3-ASR 0.6B converted to Core ML so the whole model runs on the Apple Neural Engine of Apple silicon Macs: the audio encoder and the text decoder. It was made for Speakey, a dictation and voice-command app for the Mac, which downloads it by itself. The weights are Qwen's and were not retrained.
This replaces raymondchins/qwen3-asr-0.6b-coreml, whose decoder runs on the GPU. That repo stays up for older Speakey builds. A 1.7B version exists too, more accurate, about twice the size and time.
| Mac | Measured against the GPU build |
|---|---|
| M4 (Mac mini, 16 GB) | A take is about 40% faster, it loads in about a second instead of 6.7 s, and it draws about 6x less power |
| M5 Max | Loads about 6x faster with a fraction of the memory; a take is slower than on that Mac's large GPU, still well under what a person notices |
The GPU build compiles its Metal graph on every launch, and while it does the process briefly holds about 2 GB of memory, a lot on an 8 GB Mac. The Neural Engine build is compiled once per Mac and then loads from Core ML's cache.
| File | Size | What it is |
|---|---|---|
encoder.mlmodelc |
356 MB | Audio encoder, fp16, one 8-second window per call |
decoder-ane-a.mlmodelc |
211 MB | Decoder layers 0 to 13, int8; functions main (64 rows) and step (1 row) |
decoder-ane-b.mlmodelc |
360 MB | Decoder layers 14 to 27 and the LM head, int8; functions main, tail and step |
embeddings.i8 |
148 MB | Token embeddings, int8, 151,936 x 1024 |
embeddings.scale |
< 1 MB | One float32 scale per embedding row |
vocab.json |
2.6 MB | Qwen's byte-level BPE vocabulary |
mel.bin |
100 KB | 128 x 201 float32 mel filterbank (Whisper, Slaney) |
speakey.json |
4 KB | Special token ids, the prompt around the audio, and the language X<asr_text> ids for each language |
LICENSE, NOTICE |
Apache 2.0, and what was changed |
Total download: about 1.1 GB. Requires macOS 15 or later (Core ML state) on Apple silicon.
hf download raymondchins/qwen3-asr-0.6b-coreml-ane --local-dir qwen3-asr-0.6b-coreml
MLState.Core ML's compute plan puts all but 5 of the decoder's 2,303 operations on the Neural Engine; the 5 are the LM head's argmax.
Load the compiled models with MLModel(contentsOf:) and computeUnits = .cpuAndNeuralEngine, one MLModel per function (MLModelConfiguration.functionName). Make one state from decoder-ane-a's main and one from decoder-ane-b's main, and pass them to every function of the same file.
mel.bin as the filterbank, log10, floor at max - 8, then (x + 4) / 4), with no padding to 30 seconds.mel (1, 128, 800) and an additive key mask mask (1, 1, 1, 104); output audio (1, 104, 1024). A full 100-frame chunk gives 13 tokens, a partial chunk of n frames gives ceil(n / 8); mask the rest with -1e4. The original encoder never attends across windows, so this is exact for any length.prefix from speakey.json, the audio tokens, then suffix. Append a language's ids to force it; without them the model writes language X<asr_text> first. Text embeddings are rows of embeddings.i8 times their scale.at..<at+S and takes place (1, 1, 512, S), a one-hot with place[at+j, j] = 1 for each real row j, placeT, the same transposed to (1, 1, S, 512), cos and sin (1, 1, 128, S) for RoPE with theta = 1,000,000, and mask (1, 1, S, 512), additive over cache rows (0 up to the row itself, -1e4 after). Padding rows get a zero place column, so they write nothing.decoder-ane-a: embeds (1, S, 1024) in, hidden_out (1, 1024, 1, S) out. Pass that to decoder-ane-b as hidden.decoder-ane-b: main for pieces where no next token is needed, tail with sel (1, 64, 16) to get the next token after up to 16 chosen rows (one-hot sel[row, t]), step with sel (1, 1, 1) = 1 for one token. Outputs vals and idx, each (1, 4, 1, T): the token is idx[part] + part x 37,984 for the part with the largest value. Stop at 151643 or 151645.Checking a whole draft in one tail call costs about what one step does, which is how Speakey confirms its live-preview text when a take ends. Speakey's runtime for it is under 200 lines of Swift.
Speakey's built-in bench, warm, against the GPU build of the same weights (int8, same encoder). p50 / p90 where two numbers are given; bold is the better one.
M4 Mac mini, 16 GB, macOS 27.2, 22 dictation takes of 1.6 to 38 seconds, Indonesian and English, each decoded 4 times plain and 4 times with a draft:
| GPU build | Neural Engine build | |
|---|---|---|
| Final take with a live-preview draft | 184 / 506 ms | 106 / 228 ms |
| Final take, plain greedy decoding | 1,052 / 1,787 ms | 558 / 1,029 ms |
| Load, warm-up included (after the first) | 6.7 s | 0.8 s |
| Peak memory footprint | 373 MB | 164 MB |
| Average power over the whole bench | 12.9 W | 2.1 W |
| Energy for the same workload | 2,746 J | 240 J |
M5 Max, macOS 27.2:
| GPU build | Neural Engine build | |
|---|---|---|
| Final take with a draft (one 7.6 s take) | 19 ms | 40 ms |
| Load, warm-up included (after the first) | 4.0 s | 0.6 s |
| Peak memory footprint | 393 MB | 96 MB |
| Highest resident memory while loading | 2.1 GB | 318 MB |
Power is the whole Mac as macmon reads it (CPU, GPU and Neural Engine), sampled every 250 ms over the bench.
The weights and the int8 quantization are the same as the GPU build, so the words are too: 16 of 22 takes came back identical on the M5 Max. Most differences were takes longer than 24 seconds, which are decoded in 24-second parts here and 40-second parts on the GPU build; the rest were single words, about as often better as worse. No word error rate against human transcripts was measured.
Apache License 2.0, the same as the original model. See LICENSE and NOTICE. Qwen3-ASR 0.6B is Copyright (c) Alibaba Cloud (Qwen team). Converted and modified by Raymond Chin for Speakey.
@article{qwen3asr,
title = {Qwen3-ASR Technical Report},
author = {Qwen Team},
journal = {arXiv preprint arXiv:2601.21337},
year = {2026}
}
decoder-ane-b.mlmodelc/weights/weight.bin 377 MBencoder.mlmodelc/weights/weight.bin 373 MBdecoder-ane-a.mlmodelc/weights/weight.bin 221 MBmel.bin 102.9 kBdecoder-ane-b.mlmodelc/coremldata.bin 2.8 kBdecoder-ane-a.mlmodelc/coremldata.bin 1.8 kBencoder.mlmodelc/coremldata.bin 0.4 kBdecoder-ane-a.mlmodelc/analytics/coremldata.bin 0.2 kBdecoder-ane-b.mlmodelc/analytics/coremldata.bin 0.2 kBencoder.mlmodelc/analytics/coremldata.bin 0.2 kBQwen/Qwen3-ASR-0.6B