Boulesis-26B-A4B — GGUF (imatrix (i1) quants)

🧬 Source model: SubMaroon/Boulesis-26B-A4B · base: google/gemma-4-26B-A4B-it
📦 Static quants: spiritfather/Boulesis-26B-A4B-GGUF
🛠️ Runs on llama.cpp (master)
📊 Scored on CaliperBench
💬 Discord

[!Note] All credit for the model goes to SubMaroon, who produced Boulesis-26B-A4B — an experimental Gemma-4-26B-A4B RP merge that splices attention circuits (QK routing from Pantheon-Reasoning-1.1, a transplanted style head from StyleTune-V2, a narrative RP LoRA) onto the coder3101 heretic base. These are quantizations only.

[!Tip] Benchmarked on CaliperBench — a creative-writing benchmark scoring prose craft, roleplay and willingness rather than general intelligence. See this model's scores: caliperbench.com

[!Important] These GGUFs embed the fixed Gemma-4 chat template. The author's shipped template omits the thought-channel prime on the thinking path, so thinking mode returns empty content on stock llama.cpp; the embedded one is corrected. Serve with --jinja and pass --no-reasoning-preserve for multi-turn thinking, or the model re-reads its own earlier thoughts and answers with reasoning only.

imatrix (importance matrix) computed over bartowski calibration_datav3 on this model's own bf16. Weighted quants spend their bits where the calibration text says the model is sensitive; they are usually preferable to a static quant of the same size below Q5.

Provided quants

Loading (any current llama.cpp)

llama-server -m Boulesis-26B-A4B.i1-Q4_K_M.gguf -ngl 99 -fa on -c 32768 -ctk q8_0 -ctv q8_0 --jinja --no-reasoning-preserve

Thinking: on by default. Turn off per request with chat_template_kwargs: {"enable_thinking": false} or serve with --reasoning off --reasoning-budget 0. With thinking on, cap it per request with thinking_budget_tokens (1536 is plenty for roleplay).

Quant recipe

bf16 GGUF from the safetensors via convert_hf_to_gguf.py (llama.cpp b10819); the author's repo ships a placeholder tokenizer.json, so the base google/gemma-4-26B-A4B-it tokenizer (shared by the whole 26B-A4B family) was used for the conversion. Plain llama-quantize from that bf16 — no requantizing from a lower quant. The i1 set uses an imatrix computed on the same bf16 over bartowski calibration_datav3.

License

Gemma Terms of Use, inherited from the base model.