Qwen3.8-27B MXFP4 GGUF

GGUF quantization of Qwen/Qwen3.8-27B for llama.cpp inference on NVIDIA Blackwell GPUs.

Following the widespread community adoption of the Qwen3.5 and Qwen3.6 series, Qwen3.8 is the most capable generation in the Qwen open-model family to date. Built on the architectural foundation of Qwen3.5, Qwen3.8 delivers substantial gains across coding, professional work, research, and long-horizon agentic tasks.

Qwen3.8-27B brings these advances to a compact, deployment-friendly dense model: a native vision-language model that understands images and videos, with flexible thinking control, designed to carry complex, multi-step tasks through to completion with greater reliability.

Qwen3.8 Highlights

Core Capabilities: Comprehensive improvements across coding, professional work, research, and long-horizon agentic tasks.
Agent Execution: Stronger autonomous planning and better handling of environment feedback, leading to more reliable end-to-end task completion.
Downstream Compatibility: Broader support for popular harnesses and development tools, making it easier to integrate into your existing stack.
Flexible Thinking Control: Thinking mode is on by default and can be disabled per request; reasoning depth can be tuned with reasoning_effort, and reasoning context from historical messages is retained via preserve_thinking.
Vision-Language Understanding: Native support for image and video understanding, from STEM diagrams and documents to hour-scale videos.

Model Overview

Property	Value
Base Model	Qwen/Qwen3.8-27B
Architecture	Qwen3_5ForConditionalGeneration (vision + language)
Language Model Parameters	27B
Hidden Dimension	5120
Token Embedding	248,320 (Padded)
Number of Layers	64
Hidden Layout	16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))
Gated DeltaNet Heads	48 for V, 16 for QK (head dim 128)
Gated Attention Heads	24 for Q, 4 for KV (head dim 256, RoPE dim 64)
Feed Forward Intermediate Dim	17,408
Context Length	262,144 natively (extensible to 1M)
MTP	Included (next-token prediction head)
License	Apache-2.0

Quantization Details

Property	Value
Quantization Format	MXFP4 (OCP Microscaled FP4)
Bits Per Weight	4.36 BPW
Original Model Size	~52 GB (F16)
Quantized Size	~14.2 GB
KV Cache (recommended)	Q8_0
Quantized With	llama.cpp llama-quantize (MXFP4 ftype)

What is MXFP4?

MXFP4 uses the OCP (Open Compute Project) microscaled FP4 format with E8M0 power-of-two scaling per 32 values. The power-of-two scaling means dequantization is just bit-shifts — no floating-point multiply needed — making decode significantly faster than NVFP4 on consumer Blackwell GPUs.

MXFP4 has 50% less scale metadata per weight (0.25 bits overhead) compared to NVFP4 (0.5 bits overhead), further reducing memory bandwidth pressure during decode.

MXFP4 is the recommended format for consumer RTX 50-series GPUs due to its superior decode speed and smaller file size.

Files

File	Size	Description
`qwen3.8-27b-mxfp4.gguf`	~14.2 GB	Text model (MXFP4 quantized)
`mmproj-qwen3.8-27b-f16.gguf`	~928 MB	Vision projector (F16)

Hardware Requirements

GPU: NVIDIA RTX 50-series (Blackwell) with 16+ GB VRAM
RAM: 16+ GB system RAM recommended
Storage: ~20 GB free disk space

Usage with llama.cpp

Text Only

llama-cli -m qwen3.8-27b-mxfp4.gguf -ngl 99 -p "Hello"

With Vision

llama-cli -m qwen3.8-27b-mxfp4.gguf   --mmproj mmproj-qwen3.8-27b-f16.gguf   -ngl 99   --chat-template chatml   --conversation

Server Mode (Recommended)

llama-server   -m qwen3.8-27b-mxfp4.gguf   --mmproj mmproj-qwen3.8-27b-f16.gguf   -ngl 99   --ctx-size 32768   -ctk q8_0 -ctv q8_0   -fa on   -b 512   --ubatch-size 128

Recommended Settings (RTX 5060 Ti 16GB)

Setting	Value	Why
`-ngl 99`	All layers on GPU	Full GPU offload for speed
`--ctx-size 32768`	32K context	Sweet spot for speed/quality
`-ctk q8_0 -ctv q8_0`	Q8 KV cache	Near-lossless, 2x VRAM savings vs F16
`-fa on`	Flash attention	Faster attention kernels
`-b 512`	Batch size	Optimal for single-user
`--ubatch-size 128`	Micro-batch	Balances throughput and latency

Performance (RTX 5060 Ti 16GB)

Metric	Value
Prompt Processing	~106 t/s
Generation	~25 t/s
VRAM Usage	~15.9 GB

Context Length vs Speed

Context	Generation Speed
8K	~26 t/s
32K	~25 t/s
64K	~10 t/s

Why MXFP4 Over NVFP4?

Property	MXFP4	NVFP4
Generation Speed	~25 t/s	~10 t/s
Prompt Processing	~106 t/s	~30 t/s
File Size	14.2 GB	15 GB
Scale Type	E8M0 power-of-two	E4M3 FP8
Scale Overhead	0.25 bits/weight	0.5 bits/weight
Best For	Consumer GPUs	Datacenter Blackwell

Citation

@misc{qwen38,
    title = {{Qwen3.8-Max}: A New Bar for Coding and Cowork},
    url = {https://qwen.ai/blog?id=qwen3.8},
    author = {{Qwen Team}},
    month = {August},
    year = {2026}
}

License

Apache-2.0 (same as base model)

FreedomAISVR/Qwen3.8-27B-MXFP4-GGUF