EXL3 quants of gemma-4-12B-it
mul1 quants (recommended, improved performance on exllamav3>=v1.0.0):
3.00 bits per weight
3.50 bits per weight
4.00 bits per weight
4.50 bits per weight
5.00 bits per weight
6.00 bits per weight
8.00 bits per weight / H8
MCG quants (legacy):
2.00 bits per weight
2.50 bits per weight
3.00 bits per weight
3.50 bits per weight
4.00 bits per weight
4.50 bits per weight
5.00 bits per weight
6.00 bits per weight
8.00 bits per weight / H8