Qwen3.8-27B-MLX

An MLX build of the official Qwen3.8-27B — 4 / 6 / 8-bit + MTP drafter for Apple Silicon

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website ·

Model Catalog ·

Model API ·

GitHub ·

Discord ·

An MLX build of the official Qwen/Qwen3.8-27B

— a 27B dense hybrid-attention (Gated DeltaNet linear + full attention) native vision-language model

with thinking control, tool-calling and an MTP head — quantized to MLX for Apple Silicon.

Three precisions (4 / 6 / 8-bit, affine) as subfolders, the 4-bit build also mirrored at

the repo root (so orcarouter/Qwen3.8-27B-MLX loads directly in LM Studio), plus a separate **mtp/

drafter for speculative decoding. The vision tower, norms and conv layers stay in BF16**; only

language-model linear weights (incl. embed_tokens / lm_head) are quantized. Browse all models in

the OrcaRouter Model Catalog; deployed as API

here.

Available quantizations

|---|---|---|---:|---|---|

| 8-bit/ | 8 | 32 | ~31 GB | 32–64 GB | Near-lossless — best quality |

| 6-bit/ | 6 | 32 | ~24 GB | 32 GB | Excellent balance |

| 4-bit/ | 4 | 64 | ~16 GB | 24 GB | Very good — recommended default |

| mtp/ | bf16 | — | ~0.9 GB | — | MTP drafter (speculative decoding) |

Repo root = 4-bit/. --model orcarouter/Qwen3.8-27B-MLX (no subfolder) resolves to 4-bit.

Quality — KL-divergence vs BF16

Measured on this build: KL divergence and Top-1 token agreement of each quantized precision

against the full-precision BF16 reference (wikitext-2, 1024 tokens, MLX CUDA backend). Lower KLD /

higher Top-1 = closer to full precision.

|---|---:|---:|---:|

| 8-bit | 0.00068 | 0.00233 | 98.34 % |

| 6-bit | 0.00216 | 0.00795 | 96.97 % |

| 4-bit | 0.02824 | 0.10857 | 92.68 % |

8-bit / 6-bit are near-lossless (Top-1 ≈ 97–98 %); 4-bit stays very close (Top-1 ≈ 93 %) — the

recommended default for everyday use.

Multi-Token Prediction (MTP) — speculative decoding

Qwen3.8-27B has a native MTP head. In MLX it is loaded as a separate drafter: the main model

is loaded with MTP stripped, and the drafter is passed explicitly. The drafter lives in

mtp/ (model_type: qwen3_5_mtp, block_size 3) and works with any main precision.

hf download orcarouter/Qwen3.8-27B-MLX --include "6-bit/*" "mtp/*" --local-dir ./Qwen3.8-27B-MLX

python -m mlx_vlm generate \
    --model       ./Qwen3.8-27B-MLX/6-bit \
    --draft-model ./Qwen3.8-27B-MLX/mtp \
    --draft-kind mtp --draft-block-size 4 \
    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

Requirements: an mlx-vlm build with the qwen3_5_mtp drafter and --draft-kind mtp (mlx-vlm

main). MTP acceptance is lossless — greedy output is identical, just fewer forward passes.

Usage (mlx-vlm, Apple Silicon)

pip install -U mlx-vlm    # needs mlx-vlm >= 0.6.13, mlx >= 0.32

hf download orcarouter/Qwen3.8-27B-MLX --include "4-bit/*" --local-dir ./Qwen3.8-27B-MLX

# text
python -m mlx_vlm generate --model ./Qwen3.8-27B-MLX/4-bit \
    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256
# vision (image + text)
python -m mlx_vlm generate --model ./Qwen3.8-27B-MLX/4-bit \
    --image path/to/image.png --prompt "Describe this image." --max-tokens 256
# OpenAI-compatible server
python -m mlx_vlm server --model ./Qwen3.8-27B-MLX/4-bit --port 8080

On Apple Silicon the Metal backend is used automatically. (On a Linux CUDA backend, vision requires

MLX_CUDA_USE_CUDNN_SDPA=0; not needed on macOS.)

LM Studio

Repo root is the 4-bit build. Two things: turn off KV-cache quantization (unsupported on this

vision arch — load fails otherwise), and update the MLX runtime (Settings → Runtime) — qwen3_5

support landed in mlx-vlm 0.6.x.

Qwen3.8-27B

Following the widespread community adoption of the Qwen3.5 and Qwen3.6 series, Qwen3.8 is the most

capable generation in the Qwen open-model family to date. Built on the architectural foundation of

Qwen3.5, Qwen3.8-27B is a compact, deployment-friendly dense model: a native vision-language model that

understands images and videos, with flexible thinking control, designed to carry complex, multi-step

tasks through to completion with greater reliability.

Qwen3.8 Highlights

Core Capabilities: comprehensive improvements across coding, professional work, research, and

long-horizon agentic tasks.

Agent Execution: stronger autonomous planning and better handling of environment feedback.
Downstream Compatibility: broader support for popular harnesses and development tools.
Flexible Thinking Control: thinking on by default, disable per request; tune depth with

reasoning_effort, retain reasoning context via preserve_thinking.

Vision-Language Understanding: native image and video understanding, from STEM diagrams and

documents to hour-scale videos.

Model Overview

Type: Causal Language Model with Vision Encoder
Parameters: 27B · Hidden: 5120 · Layers: 64 · Vocab: 248,320 (padded)
Hidden layout: 16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))
Gated DeltaNet: 48 V heads / 16 QK heads, head dim 128
Gated Attention: 24 Q heads / 4 KV heads, head dim 256, RoPE dim 64
FFN intermediate: 17,408
MTP (Multi-Token Prediction): trained with multiple steps (shipped as the mtp/ drafter)
Context length: 262,144 native, extensible to 1,000,000 tokens

Best Practices

Sampling — thinking mode: temp=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0
Sampling — non-thinking: temp=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5
Output length: allow generous max-tokens; give ≥ 2048 headroom so the final answer isn't

truncated by the thinking budget.

Ultra-long context: for total length beyond 262,144, use RoPE scaling (e.g. YaRN).

Citation

@misc{qwen38,
    title  = {Qwen3.8: A New Bar for Coding and Cowork},
    url    = {https://qwen.ai/blog?id=qwen3.8},
    author = {{Qwen Team}},
    month  = {August},
    year   = {2026}
}

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B.

Quantization does not change the underlying license obligations.