Qwen3-4B-Instruct-2507 — GGUF (iPhone-optimized)
A Q4_K_M GGUF of Qwen/Qwen3-4B-Instruct-2507 for on-device inference on 8 GB+ iPhones, iPads, and Apple Silicon Macs via llama.cpp or apps that wrap it (e.g. Haplo).
Hosted by jc-builds for the Haplo ecosystem. Quantization by Unsloth. Original weights © Alibaba Cloud, redistributed under the Apache 2.0 License; this file is a quantized (modified) version.
TL;DR
The July 2025 refresh of Qwen3-4B, released as a dedicated non-thinking instruct model: it answers directly and never emits <think> blocks, which makes it a clean fit for tool-driven agents that use plain ChatML. Strong at code for its size. See the upstream model card for benchmarks.
Available quantizations
| File | Size | Recommended use |
|---|---|---|
Qwen3-4B-Instruct-2507-Q4_K_M.gguf |
2.50 GB | Default — 8 GB+ devices |
Details
| Parameters | 4.0B (3.6B non-embedding) |
| Architecture | qwen3 |
| Quantization | Q4_K_M |
| Chat format | ChatML, non-thinking only |
| Minimum device | 8 GB RAM (iPhone 15 Pro / 16 class and newer) |
How to use
Download URL:
https://huggingface.co/jc-builds/Qwen3-4B-Instruct-2507-GGUF/resolve/main/Qwen3-4B-Instruct-2507-Q4_K_M.gguf
llama.cpp
llama-cli -hf jc-builds/Qwen3-4B-Instruct-2507-GGUF:Q4_K_M
License
Apache 2.0 (see LICENSE). Qwen3 by Alibaba Cloud — see the upstream license.