Qwen3-ForcedAligner-0.6B — CoreML INT8

CoreML conversion of Qwen/Qwen3-ForcedAligner-0.6B with INT8 kmeans palettization on the two transformer bundles, for Apple Silicon.

Predicts word-level timestamps for audio + text pairs in a single non-autoregressive forward pass.

Files

The embed-tokens table is shipped as a raw fp16 binary rather than a CoreML package — the Swift runtime memory-maps the file and gathers rows with vImageConvert_Planar16FtoPlanarF, which costs ~0.5 ms per alignment instead of the ~70 ms an mlpackage round-trip would cost. Keeping the embedding at fp16 (not palettized) preserves classify-time accuracy across the full vocabulary; the palettization budget is spent on the two large transformer bundles, where it matters most for disk and ANE memory.

Usage (Swift via speech-swift)

import Qwen3ASR

let aligner = try await CoreMLForcedAligner.fromPretrained(
    modelId: "aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-INT8")
let aligned = try aligner.align(
    audio: pcmSamples,
    text: "Can you guarantee that the replacement part will be shipped tomorrow?",
    sampleRate: 16000,
    language: "English")

CLI: speech align audio.wav --engine coreml --aligner-model aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-INT8

Historical performance (M2 Max, 64 GB, debug build, 20 s clip)

Per-stage profile (COREML_ALIGN_PROFILE=1):

mel=150ms  encoder=78ms  embedding=0.5ms  splice=0ms  decoder=54ms  argmax=7ms

The Swift decoder defaults to .cpuAndGPU; the encoder defaults to .cpuAndNeuralEngine. Explicit .cpuOnly uses the separate CPU-compatible bundle below.

Architecture

Same model as the upstream Qwen3-ForcedAligner. Inference is non-autoregressive: one forward pass over a chat-template prompt (system + audio + assistant + word<timestamp><timestamp> slots), then argmax at the timestamp positions, then LIS monotonicity correction.

The causal mask is baked into the exported graph as a constant with a finite -1e4 fill value (not -inf) so the fp16 softmax cannot produce NaN.

Variants

• This repo: INT8 — kmeans-palettized text decoder + audio encoder, ~880 MB, similar RTF, ~35% lower peak memory than FP16.
• Companion: aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-FP16 — full precision, ~1.7 GB, highest fidelity.

For pure MLX paths (no CoreML), see the 4bit, 5bit, 8bit, and bf16 siblings under the same HuggingFace org.

CPU-compatible decoder

text_decoder_cpu_v1.mlmodelc and text_decoder_cpu_v1.mlpackage are additional decoder bundles for CPU-only execution. Their 113 RMS-normalization calculations use Float32; learned weights and the existing model input/output types are preserved. The original text_decoder bundles remain available for GPU execution.

The legacy decoder returns all-zero logits with CPU-only execution on macOS 15.8.2. The Float32-normalization INT8 graph matches all 22 corrected word boundaries in the reference fixture on that runtime. The release validation manifest records the supported checks; validation on other devices is separate.

Updated speech-swift loaders select the versioned bundle for explicit .cpuOnly execution. Run one online model load to populate it; an existing legacy cache does not contain the new CPU bundle. Local/custom exports can provide the same bundle name to select it. The versioned bundle must not be changed in place; future graph changes require a new bundle version.

CPU release validation

Both versioned INT8 and FP16 CPU bundles match all 22 corrected reference word boundaries on macOS 15.8.2 (24H232), with finite, nonzero logits. The checks use the packaged model files at immutable commits; see the verification run. Local M5 Pro/macOS 26.6.2 CPU and existing GPU paths also match the reference boundaries. Physical-iPhone validation remains separate. The historical performance figures above predate these CPU bundles and do not describe CPU-only execution.

Neural Engine-compatible decoder

text_decoder_ane_v1.mlmodelc and text_decoder_ane_v1.mlpackage add a decoder for .cpuAndNeuralEngine and .all. The graph uses mul(x, x) instead of pow(x, 2) in 113 RMS-normalization chains. Learned weights, FP16 precision, and model input/output types are preserved. The original GPU decoder and the separate CPU-compatible decoder remain unchanged.

The original decoder can produce incorrect timestamp classes on the Neural Engine on older runtimes. See the graph repair and affected-device results. The local release validation manifest records checks for these assets; validation on a newer Mac does not establish correctness on older devices.

Updated speech-swift loaders request this versioned bundle for explicit .cpuAndNeuralEngine or .all execution of the INT8 and FP16 models. The default remains .cpuAndGPU; .cpuOnly uses text_decoder_cpu_v1. Run one online model load to populate the new bundle. A legacy offline cache without it fails rather than using the original decoder on the Neural Engine. Local/custom exports can provide the versioned bundle under the same name. Future graph changes require a new bundle version.