Core ML conversion of Qwen/Qwen3-ASR-0.6B-hf for local speech recognition with FluidAudio on Apple platforms.
The repository contains two locally validated variants. Both run fully on-device after the files have been downloaded.
| Directory | Audio encoder | Decoder | Embedding | Compiled size | Status |
|---|---|---|---|---|---|
fp16-int8/ |
FP16 | Linear Int8 | FP16 | 1226.18 MiB | Validated reference/fallback |
lut8-int8/ |
LUT8 weights, FP16 compute | Linear Int8 + LUT8 lm-head | FP16 | 1048.56 MiB | Validated, recommended smaller variant |
fp16-int8 is the renamed baseline. lut8-int8 uses a LUT8 audio encoder and, since 2026-09-25, a reworked decoder that fixes three GPU/MPSGraph defects in the original build (see below). The embedding table and vocabulary are byte-identical between the variants.
The original lut8-int8 audio encoder attended over a single 100-frame (1 s) mel window per call, eight times narrower than the source model's inference window (n_window_infer=800). The re-exported encoder keeps the per-100-frame convolution and per-chunk positional embedding, but runs self-attention over 8 chunks (104 tokens) at once with an explicit key_mask for the padded tail. Its valid outputs match Qwen3ASRForConditionalGeneration.get_audio_features (mean abs diff 4.9e-4 after LUT8 + FP16).
| FLEURS, first 100 utterances | single-window encoder | 8 s window encoder | PyTorch reference |
|---|---|---|---|
cmn_hans_cn CER (digits and Latin removed) |
3.03% | 2.44% | 2.44% |
en_us WER |
5.20% | 4.45% | 4.54% |
Inputs are mel_input [1, 128, 800] (zero-padded) and key_mask [1, 104] (0 for the first post_cnn_length(valid frames) tokens of each 13-token chunk, -1e4 otherwise). The runtime must call the encoder once per 800 frames and keep only the unmasked outputs; this is implemented in the GotIt FluidAudio fork (github.com/Mars-Sea/FluidAudio, tag 0.17.7). Upstream FluidAudio 0.15.x only feeds mel_input and cannot load this encoder; use the pinned revision of the previous release for it. The decoder, embedding table and vocabulary are unchanged.
The original decoder loaded and transcribed correctly with CPU compute units, but the GPU compute path (.cpuAndGPU and FluidAudio's default .all) was broken on iOS 27 / macOS 26 in three ways:
position_cos/position_sin inputs carried their own flexible dimensions; models with 8+ layers never finished loading on GPU. The reworked decoder embeds the RoPE tables as constants and derives positions from hidden_states, removing those symbols.read_state after slice_update returned pre-update values, and shape-derived slice bounds wrote K/V to the wrong slots, producing garbage transcripts. The reworked decoder derives the write position from the attention mask values and places new K/V with a one-hot matmul, writing the state whole.lm_head dequantized incorrectly on GPU only (CPU was unaffected), producing logits up to ~20 apart. The lm_head is now palettized to a k-means LUT8 (same file size), which computes correctly on all backends.The reworked decoder was validated on Mac with FluidAudio 0.15.2 across .cpu, .cpuAndGPU, and .all compute units: traced-vs-eager parity max difference 0.0, CPU-vs-GPU max logit difference 0.6 (fp16 noise), and identical transcripts on synthetic Chinese and English clips on all three configurations. The CPU-only path remains the fastest to load.
cpuAndNe (.cpuAndNeuralEngine) is not supported by the stateful decoder on either layout: the Core ML runtime returns error -14 for flexible-shape stateful graphs on the ANE planner. Use .cpu, .cpuAndGPU, or .all.
Linear Int8 was not used for the audio encoder because it crashed in the GPU/MPSGraph path selected by FluidAudio's default .all compute units on the test Mac. LUT8 completed that path successfully. Int4 and mixed-Int4 experiments were rejected by application testing and are not included.
The stateful decoder requires the newer Core ML state API. Intel Macs were not tested.
When selecting compute units, prefer .cpu, .cpuAndGPU, or .all. Avoid .cpuAndNe: the stateful decoder's flexible-shape graph is rejected by the ANE planner with error -14.
fp16-int8/
qwen3_asr_audio_encoder_v2.mlmodelc/
qwen3_asr_decoder_stateful.mlmodelc/
qwen3_asr_embeddings.bin
vocab.json
metadata.json
SHA256SUMS
lut8-int8/
qwen3_asr_audio_encoder_v2.mlmodelc/
qwen3_asr_decoder_stateful.mlmodelc/
qwen3_asr_embeddings.bin
vocab.json
metadata.json
SHA256SUMS
metadata.json
Only compiled .mlmodelc bundles are included. Duplicate .mlpackage conversion sources and Transformers sidecars are intentionally omitted because FluidAudio loads the compiled bundles directly. The conversion workflow is based on the Qwen3-ASR Core ML work in FluidInference/mobius, with local adaptations for the Transformers-native checkpoint and the published variants.
Choose one variant directory and load it explicitly:
import FluidAudio
let modelDirectory = URL(
fileURLWithPath: "/path/to/qwen3-asr-0.6b-hf-coreml/lut8-int8"
)
let manager = Qwen3AsrManager()
try await manager.loadModels(from: modelDirectory, computeUnits: .all)
let samples = try AudioConverter().resampleAudioFile(
URL(fileURLWithPath: "/path/to/audio.wav")
)
let text = try await manager.transcribe(
audioSamples: samples,
language: "zh",
maxNewTokens: 512
)
print(text)
Use fp16-int8 in the path to select the reference variant. Omit the language hint to use automatic language detection. Audio is resampled to mono 16 kHz by FluidAudio.
Both published directories were loaded independently with FluidAudio 0.15.2 and .all compute units and completed end-to-end transcription.
| Check | fp16-int8 |
lut8-int8 |
|---|---|---|
| Encoder max absolute difference vs. PyTorch | 0.0039731 | 0.0046397 |
| Encoder mean absolute difference vs. PyTorch | 0.0003790 | 0.0005188 |
| Decoder prefill top-10 overlap | 7/10 | 7/10 |
| Decoder prefill-then-decode top-10 overlap | 6/10 | 6/10 |
On a 12.36-second synthetic Chinese/English smoke clip, the two variants produced identical transcripts. Recorded inference times were 5.287 seconds for fp16-int8 and 5.454 seconds for lut8-int8 during the comparison run. Additional local application testing accepted only these two variants.
For the reworked lut8-int8 decoder (2026-09-25), FluidAudio end-to-end transcription was validated on .cpu, .cpuAndGPU, and .all compute units with identical transcripts:
| Check | lut8-int8 (reworked decoder) |
|---|---|
| Decoder traced-vs-eager max absolute difference | 0.0 |
| Decoder CPU-vs-GPU max logit difference | 0.6 |
FluidAudio end-to-end transcripts, .cpu / .cpuAndGPU / .all |
identical, correct |
Validation environment:
These are conversion checks and smoke tests, not a representative WER/CER evaluation. Accuracy across the upstream model's full language and dialect set has not been established for these Core ML variants.
cpuAndNe compute units are unsupported for the stateful decoder (Core ML error -14); use .cpu, .cpuAndGPU, or .all.Qwen/Qwen3-ASR-0.6B-hf7f1569a48a89f3e3f4dc3a5c9d28bddd903bc76cmodel.safetensors SHA-256: d3f212dd20abecd315d830bc54ae3865e56ebfc3276484e57b771288ba27fd35SHA256SUMSThe upstream model is distributed under the Apache License 2.0. Review the upstream model card for intended-use guidance and limitations.
@article{Qwen3-ASR,
title={Qwen3-ASR Technical Report},
author={Xian Shi and Xiong Wang and Zhifang Guo and Yongqi Wang and Pei Zhang and Xinyu Zhang and Zishan Guo and Hongkun Hao and Yu Xi and Baosong Yang and Jin Xu and Jingren Zhou and Junyang Lin},
journal={arXiv preprint arXiv:2601.21337},
year={2026}
}
本仓库提供两个纯本地 Core ML 版本:fp16-int8 是重命名后的稳定基线,lut8-int8 是体积更小的推荐版本。lut8-int8 的解码器于 2026-09-25 重构,修复了原版本在 GPU 路径上的三个缺陷(加载卡死、KV 缓存写错位置、线性 Int8 lm_head 在 GPU 上输出错误 logits),已在 .cpu、.cpuAndGPU、.all 三种算力配置下通过 FluidAudio 0.15.2 端到端验证。注意 cpuAndNe(CPU+神经引擎)不受支持,会返回 -14 错误。其他 Int4、混合 Int4 和线性 Int8 Encoder 版本均未发布。
2026-10-11 起 lut8-int8 的音频编码器改为 8 秒注意力窗口:仍按每 100 帧分块卷积,但 8 块(104 个 token)一起做注意力,与原模型 n_window_infer=800 一致,尾部不足部分用 key_mask 屏蔽。FLEURS 前 100 条:中文 CER(去数字与外文)3.03% → 2.44%,英文 WER 5.20% → 4.45%,与 PyTorch 原模型持平。新编码器需要 GotIt 的 FluidAudio fork(Mars-Sea/FluidAudio 标签 0.17.7)按 800 帧调用并传入 key_mask,官方 FluidAudio 0.15.x 无法加载;解码器、嵌入表与词表未变。
fp16-int8/qwen3_asr_decoder_stateful.mlmodelc/weights/weight.bin 597 MBlut8-int8/qwen3_asr_decoder_stateful.mlmodelc/weights/weight.bin 597 MBfp16-int8/qwen3_asr_audio_encoder_v2.mlmodelc/weights/weight.bin 373 MBfp16-int8/qwen3_asr_embeddings.bin 311 MBlut8-int8/qwen3_asr_embeddings.bin 311 MBlut8-int8/qwen3_asr_audio_encoder_v2.mlmodelc/weights/weight.bin 187 MBfp16-int8/qwen3_asr_decoder_stateful.mlmodelc/coremldata.bin 2.3 kBlut8-int8/qwen3_asr_decoder_stateful.mlmodelc/coremldata.bin 2.3 kBlut8-int8/qwen3_asr_audio_encoder_v2.mlmodelc/coremldata.bin 0.5 kBfp16-int8/qwen3_asr_audio_encoder_v2.mlmodelc/coremldata.bin 0.4 kBfp16-int8/qwen3_asr_audio_encoder_v2.mlmodelc/analytics/coremldata.bin 0.2 kBfp16-int8/qwen3_asr_decoder_stateful.mlmodelc/analytics/coremldata.bin 0.2 kBlut8-int8/qwen3_asr_audio_encoder_v2.mlmodelc/analytics/coremldata.bin 0.2 kBlut8-int8/qwen3_asr_decoder_stateful.mlmodelc/analytics/coremldata.bin 0.2 kB有文件?提交种子。
提交种子Qwen/Qwen3-ASR-0.6B-hf