Gemma-4-26B-A4B-Heretic-EXL3 (4.0 bpw)

ExLlamaV3 4.0 bpw long-context quantization of Luni/Gemma-4-26B-A4B-Heretic.

• VRAM Weight: ~13.2 GB (100% VRAM resident on RTX 5090 32GB)
• Host RAM Offload: 0 GB
• Context: 128k full context entirely in VRAM with FP8 cache.

model_name: Gemma-4-26B-A4B-Heretic-EXL3-4.0bpw
model_dir: /path/to/model
backend: exllamav3
gpu_split: [28]
cache_mode: fp8