EXL3 quants of DeepSeek-V4.1-Flash-1017
⚠️ Requires ExLlamaV3 v1.6.6 (or dev branch)
SC quants:
1.68 bits per weight
2.25 bits per weight
2.68 bits per weight
| quant | AIME-25 | GPQA-D |
|---|---|---|
| 1.68 bpw | 96.7% | 90.4% |
| 2.25 bpw | 100% | 90.4% |
| 2.68 bpw | 100% | 89.9% |
| baseline | 100%¹ | 90.2%² |
¹ Assumed, given quant scores, couldn't find a source
² From OpenRouter
(more quants to follow, along with more benchmarks)
If you get corrupted output on short prompts, it is likely just a failure mode of the model itself. See here.