Gemma-4-26B-A4B-Heretic-EXL3 (6.0 bpw)

ExLlamaV3 6.0 bpw maximum fidelity quantization of Luni/Gemma-4-26B-A4B-Heretic.

• VRAM Weight: ~19.5 GB (100% VRAM resident on RTX 5090 32GB)
• Host RAM Offload: 0 GB
• Context: Up to ~32k context in VRAM with FP8 cache.

model_name: Gemma-4-26B-A4B-Heretic-EXL3-6.0bpw
model_dir: /path/to/model
backend: exllamav3
gpu_split: [28]
cache_mode: fp8