💻 Gemma-4-12B-Coder (fable5 × composer2.5) — imatrix GGUF ✨

Runs anywhere llama.cpp runs — AMD/Vulkan, CPU, Apple, NVIDIA. No Blackwell, no MTP, just GGUF. 🐧🍎🪟

Importance-matrix (imatrix) quants of yuxinlu1's coding model, calibrated on real Python coding data so the low-bit builds keep their coding smarts. Text-only (a coding model — no vision baggage). 💚

---

🙏 Credit

Quants of yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1 — all thanks to @yuxinlu1 for the model. ⭐ the original and watch it for a v2! The author's recipe: a fine-tune of google/gemma-4-12B-it on execution-verified Python coding chains-of-thought (Composer 2.5 real CoT + a Fable 5 "second-attempt" set for the hard cases). It thinks in Gemma's native channel, then writes clean, runnable code. De-refused; Python/algorithmic focus; English-centric.

Why this repo: the originals are static GGUF. These add an importance matrix (code-calibrated) so IQ4_XS / Q4_K keep more quality at low VRAM — the builds that fly for AMD/Vulkan and CPU folks.

---

📦 Pick your quant (all imatrix)

💡 8 GB VRAM (or 6 GB): grab Q3_K_S (5.5 GB) — it leaves headroom for context and still codes well. On the Vulkan backend (AMD) all of these fly. ⚠️ Avoid IQ3 (i-quant 3-bit) for this model — IQ3_XXS/IQ3_S collapse to gibberish here (gemma-4's special attention layers don't survive 3-bit i-quants). The Q3_K_* K-quants stay coherent at the same size — that's why the small tiers are Q3_K, not IQ3.

---

🚀 Run it (llama.cpp — any backend)

# build llama.cpp with your backend (Vulkan for AMD):  cmake -B build -DGGML_VULKAN=ON && cmake --build build
# grab one quant:
hf download sakamakismile/gemma-4-12B-coder-fable5-composer2.5-GGUF gemma-4-12B-coder-fable5-composer2.5-IQ4_XS.gguf --local-dir .

# chat server (OpenAI-compatible at http://localhost:8080)
./llama-server -m gemma-4-12B-coder-fable5-composer2.5-IQ4_XS.gguf \
  -ngl 99 --ctx-size 16384 -fa on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 64 --host 0.0.0.0 --port 8080

⚠️ Needs a recent llama.cpp — this is the gemma4 architecture (older builds won't load it). 🧠 Thinking is on by default via the chat template (--jinja). The model reasons through edge cases, then writes the code. For deterministic coding use --temp 0.

🦙 Ollama (one line, straight from this repo)

ollama run hf.co/sakamakismile/gemma-4-12B-coder-fable5-composer2.5-GGUF:Q4_K_M

Pick any tag: Q3_K_S Q3_K_M IQ4_XS Q4_K_M Q5_K_M.

❗ "manifest not found"? You must include both the hf.co/ prefix and an explicit quant tag. Without a tag, Ollama looks for :latest (which doesn't exist here); without hf.co/, it searches Ollama's own registry instead of this repo. The fix is just …-GGUF:Q4_K_M.

Also works in LM Studio / Jan / KoboldCpp — import the GGUF, pick a quant, go. 🐾

---

📊 How good is it? (greedy pass@1)

Strong at hard algorithms, bug-fixing & refactoring, and faithful open reasoning. Japanese prompts cause no measurable Python-quality drop.

⚠️ One honest caveat: on time-series / quant-finance code it can introduce a look-ahead bias (and its reasoning may state the right rule while the code does the opposite). Great algorithm/debug helper — but review its pandas/numpy back-test code before trusting it.

---

🔧 Quant details

• imatrix computed on a code-heavy calibration set (HumanEval + MBPP problems & solutions) so the importance matrix reflects real coding activations.
• Source: the author's Q8_0 GGUF (≈lossless). Text-only gemma4 (no vision/audio).
• Higher tiers keep token-embeddings & output tensors at Q6_K (K-quant default) for fidelity where it matters most; the Q3_K tiers trade a little there for size.
• K-quants over i-quants here: gemma-4's heterogeneous attention (head_dim 256 / 512 layers) survives Q3_K_* but collapses under IQ3_* — verified, so the small tiers ship as Q3_K.

📚 License & use

Gemma Terms of Use (derivatives must comply). De-refused / not safety-aligned — add your own guardrails. Best on Python/algorithmic tasks; double-check general facts and time-series code. Shared as-is. Quants & eval by Lna-Lab; thanks to @yuxinlu1. 🐾✨