Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled MXFP4 MoE GGUF
GGUF quantization of lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled — a 35B MoE (3B active) reasoning model distilled from Claude 4.7 Opus.
Model Details
- Architecture: Qwen3.5 MoE, 41 blocks (40 layers + 1 MTP head), 256 experts (8 active/token)
- Active Parameters: ~3B
- Context: 262,144 tokens
- Source: Apache 2.0 license
- Text-only: No vision support
Quantization
| File | Type | Size | BPW |
|---|---|---|---|
qwen36-35b-distill-mxfp4_moe.gguf |
MXFP4 (experts) + Q8_0 (non-experts) | ~18.87 GB | 4.56 |
MoE expert weights quantized to MXFP4. Non-expert weights (attention, shared experts, norms) quantized to Q8_0. Router weights kept at F32.
Usage
./llama-server -m qwen36-35b-distill-mxfp4_moe.gguf --host 0.0.0.0 --port 8080
Credits
- Original model: lordx64
- Quantization: FreedomAISVR