Qwen3.5-122B-A10B PS-IQ2_XXS-MTP (Peasant Smith)
The first IQ2_XXS-class GGUF of Qwen3.5-122B-A10B — smaller than any published quant (unsloth's ladder floors at Q3_K_S 52.5 GB) — built entirely from the original Qwen/Qwen3.5-122B-A10B weights with the native MTP (nextn) head embedded at Q8_0.
File: Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP.gguf — 41.77 GB (38.9 GiB), ≈2.7 bits/param over 122B total params.
SHA-256: 01bd59623e7d4f2f45bd1ce8665042bf449b8000a76480281bef7866cfce5557
Provenance chain (100% original weights)
Qwen/Qwen3.5-122B-A10Bofficial BF16 safetensors (39 shards, 244 GB — no third-party quant or modified repo involved)- → GGUF master via llama.cpp
convert_hf_to_gguf.py(archqwen35moe,nextn_predict_layers = 1, MTP remapped toblk.48.*and embedded at full precision) - →
llama-quantizeimportance-matrix quant with the tensor split below
Imatrix: imatrix_unsloth.gguf from unsloth/Qwen3.5-122B-A10B-GGUF (359 MB, large diverse calibration corpus). Verified before use: 144/144 main expert tensors covered, zero naming mismatches against our master; the only uncovered tensors are the 3 MTP expert tensors, which were pinned out of IQ quantization.
Tensor split (verified from output file — 0 violations)
| Class | Tensors | Type |
|---|---|---|
| Routed expert gate/up | 96 | IQ2_XXS |
| Routed expert down | 48 | Q2_K |
| Token embeddings | 1 | Q6_K |
| Attention, shared experts, router, output, norms | 384 | Q8_0 |
| Layer norms/biases | 370 | F32 (kept) |
| MTP / nextn block (blk.48) | 20 | Q8_0 |
Down-projection gets one full step above gate/up because its errors land directly and additively on the residual stream, while gate/up errors pass through activations first. The MTP head is kept at maximum fidelity so the draft head stays sharp.
Measured quality checks
| Test | Result |
|---|---|
| Greedy smoke ("The capital of France is") | ✅ "The capital of France is Paris." (finish=stop) |
| Needle recall ("9137 steps", 8k ctx) | ✅ exact answer 9137 |
| LRU cache code task (8k ctx) | ✅ complete correct doubly-linked-list LRU with sentinel nodes, O(1) get/put, eviction (5000-token continuous generation, zero degradation) |
| Long-generation stability | ✅ 5000 tokens continuous at steady t/s |
| Perplexity (wikitext-2) | not measured — see honesty note below |
Usage
llama-server -m Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP.gguf -ngl 99 -ncmoe 20 -c 8192 --load-mode none
Requires a recent llama.cpp master (qwen35moe + linear-attention + MRoPE support). The model ships its chat template; use --jinja.
Limitations
- 2-bit experts: real quality delta vs BF16 exists by construction; task battery above passed but long-tail knowledge degradation is expected at this footprint
- Text-only (vision tower of the original repo is not part of this GGUF)
- Not verified beyond 8k context
- Ollama can load the file, but
-ncmoe-style expert pinning is llama.cpp-only
Running with ollama
ollama derives its tag from the GGUF filename (last dash-separated segment):
ollama run hf.co/peasantsmith/Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP-GGUF:IQ2_XXS
Notes:
- The embedded MTP/nextn head is currently inert (llama.cpp has no draft wiring for the
qwen35moearchitecture yet); the model runs as a standard A10B MoE. - Expert-offload flags like
-ncmoeare llama.cpp-only; ollama loads with its default placement. Measured ~17 t/s with llama.cpp on 3x RTX 3060 (experts partially on CPU) - see benchmarks below.
Credit
Upstream model by Qwen (Apache-2.0). imatrix by Unsloth. Quant by Peasant Smith.