Gemma-4-31B-Heretic-EXL3 (4.0 bpw)
ExLlamaV3 4.0 bpw quantization of Luni/Gemma-4-31B-Heretic.
- VRAM Weight: ~15.5 GB (100% VRAM resident on RTX 5090 32GB)
- Host RAM Offload: 0 GB
- Context: Up to ~64k context in VRAM with FP8 cache.
ExLlamaV3 4.0 bpw quantization of Luni/Gemma-4-31B-Heretic.