EXL3 quants of GLM-4.6V
⚠️ Requires ExLlamaV3 v0.0.18 (or v0.0.17 dev branch)
Base bitrates:
2.00 bits per weight
3.00 bits per weight
4.00 bits per weight
5.00 bits per weight
Optimized:
2.13 bits per weight
2.32 bits per weight
2.55 bits per weight
2.80 bits per weight
3.13 bits per weight
3.55 bits per weight
4.07 bits per weight