EXL3 quants of Qwen3.8-Flash-Next
Plain quants:
2.05 bits per weight / H4, NG4
3.05 bits per weight / H5, NG5
4.05 bits per weight / H6, NG6
5.05 bits per weight / H6, NG6
6.05 bits per weight / H6, NG6
Self-calibrated quants
1.66 bits per weight / H4, NG3 ¹
2.18 bits per weight / H6, NG8
2.68 bits per weight / H6, NG8
3.18 bits per weight / H6, NG8
3.68 bits per weight / H6, NG8
4.18 bits per weight / H6, NG8
Note ngram_embedding.safetensors is swappable between quants.
¹ 1.66 bpw quant: mostly K=1.5 expert tensors, 4 bpw lm_head and input embeddings, 3 bpw n-gram table (swappable, usually streamed from disk anyway) and 4 bpw vision tower, fits on one headless RTX 5090 without CPU offloading along with MTP head and vision, for 200+ t/s in agentic workloads (with limited context though: 64k tokens at K5V4 cache quantization.)
Some limited benchmarks (more to come, probably):
| Model | GPQA-D | ERQA |
|---|---|---|
| SC 1.66 bpw / H4, NG3, V4 | 86.4% | 62.7% |
| SC 2.18 bpw | 89.4% | 64.3% |
| SC 2.68 bpw | 91.4% | 66.5% |
| SC 3.18 bpw | 91.4% | 68.3% |
| SC 3.68 bpw | 91.9% | 68.5% |
| GSQ-RCO IQ2_XS, 2.40 bpw | 87.4%¹ | |
| GSQ-RCO IQ3_XXS, 2.90 bpw | 91.4%¹ | |
| Original | 91.7% | 68.8%² |
¹ From ISTA-DASLab release
² Tested unquantized on OpenRouter (qwen/qwen3.8-flash); Qwen reported score of 72.3% from model release does not
reproduce with evalscope.
LocalMaxxing