Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta 26A5353q, 2026-06-11).

Measured decode — iPhone 17 Pro: 72 tok/s · Mac (M4 Max): 210 tok/s (DeviceMark row qwen3.5-0.8b, int8hu bundle · data)

Qwen3.5-0.8B — Apple Core AI (.aimodel)

[!NOTE] Update 2026-07-15: gpu-pipelined-b2/ adds qwen3_5_0_8b_decode_int8hu_block32_sym re-exported with coreai-core 1.0.0b2, loadable on the OS 27 beta 3 toolchain (June-era b1 bundles fail to load there with a versioned-IR error). The original b1 tree is retained unchanged so existing apps and pinned catalogs keep working. The b2 decode bundle is the exact artifact measured on DeviceMark.

Qwen3.5-0.8B converted to Apple's Core AI (the Core ML successor announced at WWDC26), ready to run on iOS 27 / macOS 27. A hybrid linear-attention model — 3 gated-delta (Mamba-style) layers per full-attention layer — running through Core AI's runtime, greedy top-1 exact vs the Hugging Face reference.

This repo publishes one bundle per platform × compute-unit: the best verified configuration (plus the cross-platform gpu-pipelined/ bundle) — each file is the exact artifact behind the published numbers, nothing experimental.

Requires iOS 27 / macOS 27 (Core AI ships with the OS). Conversion code, knowledge base, and the Swift runner: coreai-model-zoo.

Use it

New to Core AI? Start with CoreAIKit 0.7.3. Follow its requirements and first-run steps for qwen3-0.6b, then open the same release's ChatDemo. The README records the tested OS/SDK and download size; model and device coverage is stated per example.

Measured decode — iPhone 17 Pro: 72 tok/s · Mac (M4 Max): 210 tok/s (DeviceMark row qwen3.5-0.8b, int8hu bundle · data)

⚡ One line — run the kit's task op on this model (import CoreAIOps; no session, no model plumbing, downloads on first use):

let tldr = try await CoreAI.summarize(text, options: .model("qwen3.5-0.8b"))

Every op, one shape — Cookbook.

▶️ Run it (source) — the ChatDemo runner (GUI + CLI, one app for every chat model in the catalog):

git clone --branch 0.7.3 --depth 1 https://github.com/john-rocky/coreai-kit
export DEVELOPER_DIR=/Applications/Xcode-27.0.0-RC.app/Contents/Developer
open -a /Applications/Xcode-27.0.0-RC.app coreai-kit/Examples/ChatDemo/ChatDemo.xcodeproj
# → Run, then pick "Qwen3.5 0.8B" in the model picker

# agents / headless (macOS):
cd coreai-kit/Examples/ChatDemo
swift run -c release chat-cli --model qwen3.5-0.8b --prompt "What can you do, offline?"

Use Xcode build 27A266a from the release's .xcode-pin; adjust the app path if your installation is named differently.

💻 Build with it — complete; the glue is kit API, copy-paste runs:

import CoreAIKit

let id = "qwen3.5-0.8b"
let chat: ChatSession
if id == "qwen3-0.6b" {
    // Freeze the release starter; other selections retain the live catalog's
    // model-specific dispatch (including paired Gemma bundles).
    guard let model = ModelCatalog.builtin.entry(id: id)?.modelID else {
        throw CoreAIKitError.modelNotAvailableOnPlatform(id: id)
    }
    chat = try await ChatSession(model: model)
} else {
    chat = try await ChatSession(catalog: "qwen3.5-0.8b")
}
let reply = try await chat.respond(to: prompt)
// reply: the answer, generated fully on-device

The take-home is Examples/ChatDemo/Sources/QuickStart.swift — this exact code as one typed function, no UI; the CLI is an argument shell over it, and the GUI drives the same ChatSession across turns for its transcript. Multi-turn? Hold the ChatSession and call respond(to:) per turn — it keeps the conversation history; streamResponse(to:) yields tokens as they decode.

Integration checklist

• SPM: https://github.com/john-rocky/coreai-kit (exact 0.7.3) → product CoreAIKit
• Info.plist: none needed
• Entitlements: none needed
• First run downloads the model — ~1,338 MB (Mac) / ~1,338 MB (iPhone) — then it loads from the local cache (Application Support; progress via the downloadProgress callback)
• Measure in Release — Debug is ~3× slower on per-token host work

Pick your platform (measured: iPhone 17 Pro / M4 Max, greedy, top-1 exact vs HF)

• The ★★★ ship bundle adds an untied lm_head quantized as per-block-32 absmax int8 (int8hu --head-sym): the fp16 head was 54% of the per-token weight read on the bandwidth-bound phone — quantizing it is +40% on iPhone (and +3% on M4 Max). Quantize big-vocab heads with plain absmax symmetric; the default symmetric_with_clipping clips outlier head rows and corrupts top-1s. Greedy rollouts are token-identical to the ★★ bundle; same run contract. Naming note (2026-06-11): the directory is named _perchan_sym, but its head is per-block-32 — the export script of the day parsed the granularity flag without applying it (since fixed). The numbers above were measured on exactly these bytes and stand. Genuinely per-channel (axis-0) int8 weights turned out to be broken on the OS 27 beta GPU delegate (garbage logits — delegate lowering bug, minimal repro in the zoo), so per-block-32 + symmetric IS the correct ship shape, not a stand-in. The dir name is kept to avoid breaking download paths.
• The ★★ pipelined bundle is the fastest decode on BOTH platforms, with zero custom kernels: a decode-only loop-free graph (static [1,1] query, dynamic KV) that rides Apple's coreai-pipelined engine (CoreAILanguageModels / EngineFactory — async non-blocking encode, on-GPU argmax sampling, on-device KV growth) instead of a per-token run loop. Token-for-token == the fp16-GPU sequence; 16/16 single-step top-1 vs the fp32 HF oracle. It needs two things from the zoo: the engine extra-states patch (the stock engine carries exactly 2 states; the SSM conv/rec states ride as fixed-shape extras) and COREAI_CHUNK_THRESHOLD=1 at run time (prefill = pipelined S=1 steps ≈ decode speed — so for LONG prompts the ★ static pair below still wins time-to-first-token). Export: conversion/export_qwen3_5_decode_pipelined.py.
• The ★ int8 fused-kernel monolith is the custom-kernel static config (~3× dynamic, ~1.6× the fp16 static path; the current app-release config): the device GPU is weight-bandwidth-bound, so fused dequant-in-matvec Metal kernels (embedded in the .aimodel — 100% Core AI, WWDC26 session 325) halve the per-token weight stream; the 248320-token tied head runs as a fused matvec + two-level GPU argmax (greedy). Pair it with the prefill companion: the prompt is consumed 16 tokens per pass (in-graph unrolled SSM scan, fp32 recurrence; full blocks only, remainder + generation on the decode graph). Decode output is byte-identical with and without it.
• The static monoliths are GPU-only — the fp32-SSM form does not produce correct output on the ANE on the OS 27 betas (and custom Metal kernels are GPU-only); use the ANE bundle there.
• The dynamic int8 bundle (one graph, prefill+decode, 4 states keyCache/valueCache/convState/recState) is the proven Neural-Engine path — and the same file is the best macOS config (the ios-ane/ and macos/ files are identical content; pick by folder for clarity).
• The SSM while_loop doesn't lower on device delegates — these bundles use the loop-free single-step decode (bit-identical at query_len=1; the prefill graph unrolls the same scan 16× with the state held fp32). Story + gotchas: knowledge base.
• int8 in the static/dynamic bundles is k-means palettization, gated 8/8 vs the HF oracle in PyTorch before export; the pipelined bundle uses linear per-block int8 (scale-multiply dequant — 256-entry k-means LUTs are slow on the GPU delegate, 204 vs 113 tok/s on M4 Max). int4 does not survive on this model (head/MLP/SSM all degrade; k-means g8 with int8 rescue layers also fails the oracle gate; unlike Gemma 4).

Run it (pipelined, Swift, macOS 27)

git clone https://github.com/apple/coreai-models && cd coreai-models
git apply <(curl -sL https://github.com/john-rocky/coreai-model-zoo/raw/main/apps/coreai-pipelined-extra-states.patch)
COREAI_CHUNK_THRESHOLD=1 swift run -c release llm-benchmark \
  --model <path-to>/gpu-pipelined/qwen3_5_0_8b_decode_int8lin -p 128 -g 256 -n 3

In an app, load the bundle via LanguageBundle + EngineFactory.createEngine (set COREAI_CHUNK_THRESHOLD=1 before engine creation; never call warmup() — it warms shape 256, the S=1 graph rejects it; a 1-token generate is the warmup).

Run it (Python, macOS 27)

import coreai.runtime as rt
model = await rt.AIModel.load(Path("qwen3_5_0_8b_decode_int8.aimodel"),
        rt.SpecializationOptions.from_preferred_compute_unit_kind(rt.ComputeUnitKind.gpu()))
fn = model.load_function("main")
out = await fn({"input_ids": rt.NDArray(ids), "position_ids": rt.NDArray(pos)}, state=state)

On device, push the bundle into your app sandbox (xcrun devicectl device copy to --domain-type appDataContainer) — see the Swift runtime notes. Tokenizer: use the original Qwen/Qwen3.5-0.8B tokenizer (swift-transformers loads it directly).

Parity

Greedy decode matches the HF eager reference 8/8 tokens, top-1 exact (prompt-level cosine 0.9999+), verified on macOS conversion and re-verified end-to-end on the iPhone per compute unit. The int8-kernel monolith additionally passes the chained Mac-GPU greedy 8/8 vs the oracle, and the prefill companion passes both an oracle gate and a chunked-vs-q=1 parity gate (identical tokens). ⚠️ Known issue, still present on 26A428, affecting all Core AI LLMs (and how these bundles dodge it): the KV-write bug page.

CoreML (iOS 18+) variant of this model: qwen3.5-0.8B-CoreML.

---

More models in this format: Core AI Model Zoo — 75 models, each with the recipe that produced it.

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.