Qwen3.6 35B A3B, GGUF
Qwen3.6 35B A3B is a multimodal Mixture-of-Experts large language model designed for efficient long-context reasoning and scalable deployment. It is the successor to Qwen3.5 A3B and the first open-weight Qwen3.6 variant. It features a hybrid architecture combining Gated DeltaNet and full attention. It integrates a vision encoder for unified multimodal processing.
This repository holds Vultr's GGUF quantizations of it. They are hybrid quantizations: rather than applying one precision to the whole network, each layer's precision is chosen from a measured importance score, so the bits go where they change the output most.
Model Overview
| Attribute | Value |
|---|---|
| Base model | Qwen/Qwen3.6-35B-A3B |
| Architecture | qwen35moe, Mixture-of-Experts |
| Parameters | 35B total |
| Active parameters | ~3B |
| Layers | 40 |
| Vision encoder layers | 27 |
| Hidden size | 2,048 |
| Attention heads | 16 (2 key-value heads) |
| Experts | 256 total, 8 routed and 1 shared activated per token |
| Context length | 262K tokens, extendable to 1M |
| Vocabulary | 248,320 |
| License | Apache 2.0 |
| Attribute | Value |
|---|---|
| Base model | Qwen/Qwen3.6-35B-A3B |
| Architecture | qwen35moe, Mixture-of-Experts |
| Parameters | 35B total |
| Active parameters | ~3B |
| Layers | 40 |
| Vision encoder layers | 27 |
| Hidden size | 2,048 |
| Attention heads | 16 (2 key-value heads) |
| Experts | 256 total, 8 routed and 1 shared activated per token |
| Context length | 262K tokens, extendable to 1M |
| Vocabulary | 248,320 |
| License | Apache 2.0 |
Quantizations
| File | Type | Size | KLD vs bf16 |
|---|---|---|---|
| Qwen3.6-35B-A3B-Q5_K_M.gguf | Q5_K_M | 27.19 GB | 0.0125 |
| Qwen3.6-35B-A3B-Q4_K_M.gguf | Q4_K_M | 23.26 GB | 0.0234 |
| Qwen3.6-35B-A3B-IQ3_M.gguf | IQ3_M | 16.97 GB | 0.0629 |
| Qwen3.6-35B-A3B-IQ3_S.gguf | IQ3_S | 16.77 GB | 0.0696 |
| File | Type | Size | KLD vs bf16 |
|---|---|---|---|
| Qwen3.6-35B-A3B-Q5_K_M.gguf | Q5_K_M | 27.19 GB | 0.0125 |
| Qwen3.6-35B-A3B-Q4_K_M.gguf | Q4_K_M | 23.26 GB | 0.0234 |
| Qwen3.6-35B-A3B-IQ3_M.gguf | IQ3_M | 16.97 GB | 0.0629 |
| Qwen3.6-35B-A3B-IQ3_S.gguf | IQ3_S | 16.77 GB | 0.0696 |
The vision encoder is in Qwen3.6-35B-A3B-mmproj-f16.gguf (899 MB). llama.cpp downloads it automatically with the command under Run, so image input works without extra steps.
KL divergence measures how far a quantized model's next-token probabilities drift from those of the original bf16 model, averaged over every token of the evaluation text. The lower the value, the closer the quantized model is to the original; zero would mean identical output.
Evaluation
Every variant in this repository was evaluated against the original bf16 model for perplexity, KL divergence, top-token agreement, HellaSwag and WinoGrande, under identical conditions. The full results are maintained on the model's page in the Vultr Model Library:
https://docs.vultr.com/models/alibaba-qwen-36/qwen3.6-35b-a3b?tab=quantized
That page is the reference for these files. The summary above is an excerpt from it.
Run
llama-server -hf vultr/Qwen3.6-35B-A3B-GGUF:Q4_K_M
The model's chat template is embedded in every file, so no extra prompt formatting is needed.
The repository also includes mtp-Qwen3.6-35B-A3B-Q8_0.gguf, the model's multi-token prediction (MTP) head, for optional speculative decoding with any quant above:
llama-server -hf vultr/Qwen3.6-35B-A3B-GGUF:Q4_K_M --spec-type draft-mtp
License
These files are derived from Qwen/Qwen3.6-35B-A3B and are distributed under the same license, Apache 2.0. The model is the work of Qwen; Vultr produced the quantizations.