Qwen3.6-35B-A3B Selective NVFP4 MoE-MLP GGUF

An accuracy-oriented GGUF conversion of Qwen/Qwen3.6-35B-A3B for NVIDIA Blackwell GPUs. Only the routed and shared-expert MLP weights use NVFP4 W4A16; attention, embeddings, the output head, vision components, and embedded MTP remain at BF16/F32 precision.

Files

| File | Purpose | Size |

| --- | --- | ---: |

| Qwen3.6-35B-A3B-NVFP4-W4A16-MoE-MLP-BF16attn-BF16head-MTP.gguf | Main model with embedded MTP | 24.58 GB |

| mmproj-Qwen3.6-35B-A3B-BF16.gguf | BF16 vision projector | 0.90 GB |

| Qwen3.6-35B-A3B-DFlash-BF16.gguf | Optional original DFlash speculative drafter | 0.78 GB |

The main GGUF works by itself for text. Add mmproj for image input. The DFlash GGUF is not a standalone model and is used instead of embedded MTP when DFlash speculation is enabled.

Precision layout

| Component | Precision |

| --- | --- |

| Routed/shared-expert gate, up, and down weights | NVFP4 W4A16 |

| Full and linear attention | BF16/F32 |

| Embeddings, routers, and lm_head | BF16/F32 |

| Vision and MTP/NextN | BF16/F32 |

The main GGUF contains 240 NVFP4 tensors and retains all other tensors at source precision.

Usage

A recent CUDA build of llama.cpp with Qwen3.6 NVFP4 support is required. DFlash additionally requires a compatible build such as DFlash 2 PR #27342.

Embedded MTP:

llama-server \
  -m Qwen3.6-35B-A3B-NVFP4-W4A16-MoE-MLP-BF16attn-BF16head-MTP.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-BF16.gguf \
  --ctx-size 32768 --flash-attn on --n-gpu-layers -1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-mtp --spec-draft-n-max 2

Original DFlash:

llama-server \
  -m Qwen3.6-35B-A3B-NVFP4-W4A16-MoE-MLP-BF16attn-BF16head-MTP.gguf \
  -md Qwen3.6-35B-A3B-DFlash-BF16.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-BF16.gguf \
  --ctx-size 32768 --flash-attn on --n-gpu-layers -1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-dflash --spec-draft-n-max 7 --spec-draft-ngl all \
  --cache-type-k-draft q8_0 --cache-type-v-draft q8_0

Remove --mmproj for text-only use.

Validation

Structural audits passed: 240 intended MoE tensors in NVFP4; attention, lm_head, vision, and MTP retained at BF16/F32.
The target and projector successfully allocated a 262,144-token context and completed a real image request. A near-limit prompt has not yet been tested.
DFlash completed correctness and code-generation smoke tests at 32,768 context. The longer test accepted 210 of 308 drafted tokens (68.18%) and generated at 330.61 tok/s on one RTX PRO 6000 Blackwell; this is not a controlled benchmark.

Sources and license

Target: Qwen/Qwen3.6-35B-A3B
Draft: z-lab/Qwen3.6-35B-A3B-DFlash
Quantization: NVIDIA Model Optimizer
Runtime and conversion: ggml-org/llama.cpp