Qwen3-0.6B Chat CoreML

CoreML conversion of Qwen/Qwen3-0.6B for on-device inference on Apple Silicon (iPhone, iPad, Mac).

Model Details

Usage

This model is designed for use with the speech-swift Qwen3Chat module:

import Qwen3Chat

let model = try await Qwen3ChatModel.fromPretrained()

// Single generation
let response = try model.generate(messages: [
    ChatMessage(role: .user, content: "Hello!")
])

// Streaming
let stream = model.chatStream("What is Swift?", systemPrompt: "Be brief.")
for try await chunk in stream {
    print(chunk, terminator: "")
}

Prompt caching

The chat() / chatStream() methods cache the system prompt KV state. Subsequent turns restore from cache instead of re-prefilling (~300ms saved per turn).

Files

Conversion

Converted using coremltools 9.0 from the original PyTorch weights:

python scripts/convert_qwen3_chat_coreml.py \
    --hf-model Qwen/Qwen3-0.6B \
    --output models/Qwen3-0.6B-Chat-CoreML \
    --quantize int4

KV Cache Design

The CoreML model uses explicit KV cache inputs/outputs per layer:

• Inputs: layer_{i}_key_cache, layer_{i}_value_cache (float16)
• Outputs: layer_{i}_key_cache_out, layer_{i}_value_cache_out (float16)
• Shape: [1, 8, seq_len, 128] (batch, kv_heads, sequence, head_dim)

License

Apache 2.0 (same as base model)

Links

• Library Documentation — Full API reference and guides
• Blog — Technical articles and updates
• GitHub — Source code

---

• Guide: soniqo.audio/guides/chat
• Docs: soniqo.audio
• GitHub: soniqo/speech-swift