Sign up

ProCreations/grug-27b-qat-q4-gguf

🤗 Hugging Face sourceapache-2.034 GBGGUF✓ HF checksums availableupdated today
No torrent yet →

grug-27b-qat-q4-gguf

2026-07-23 refresh: rock now QAT-trained from grug-27b v2.1 weights (deep-think + agent-discipline update). bench table below measured on the v1-era rock vs v1 control - directional guide still; v2.1 rock inherit both the QAT recovery AND the v2.1 brain improvements.

grug put 27b brain in four-bit cave DURING training. brain feel rounding rock before final squish. this not normal quant. this QAT recovery rock, made for Q4 people.

recipe (same as grug-9b-qat, scaled): full-weight QAT on grug-27b, fake int4 asymmetric group-32 with straight-through gradient, ~3M token of grug-think data, Adafactor LR 2e-6, 249 step on one H200. release = 25% QAT move + 75% original anchor (full QAT overcorrect, 9b teach grug this). then fresh BF16 export, quantize ONE time to Q4_K_M.

rocks

file what
grug-27b-qat-Q4_K_M.gguf the QAT rock, ~16.5 GB
mmproj-grug-27b-f16.gguf eye rock (vision), same as main gguf repo

number. same cave, same harness, same llama.cpp build

three rock fight: ordinary Q4 (control), full-QAT Q4, and this rock (25% QAT blend). full-QAT win MBPP big but BREAK tool hand (agent valid 96.6 -> 82.8). grug no ship broken hand. blend rock best overall:

test control Q4 full-QAT Q4 THIS ROCK
MBPP-60 pass % 81.7 91.7 88.3 (+6.6)
GSM8K-60 % 96.7 95.0 98.3 (+1.6)
agent tool-call valid % 96.6 82.8 96.6 (same)
agent right-tool % 93.1 79.3 89.7 (-3.4)
agent args schema-valid % 100 100 100
identity loop rate % 3.3 0.0 1.7 (halved)
greedy longform loops 1 0 0

QAT feel rounding rock during training -> Q4 squish hurt less. coding and math UP, loop sickness DOWN, tool hand intact. small right-tool dip is the honest trade. grug show all numbers, hide nothing.

if rock act broken

single-token spam ("/" forever etc) = NOT the rock. hybrid DeltaNet brain CANNOT survive llama.cpp context-shift: old builds shift on context overflow and corrupt the recurrent state into token spam. fix:

  • use RECENT llama.cpp (qwen3_5 support; new builds refuse instead of shift)
  • agent frontends (OpenCode etc): set -c 16384 or bigger
  • still broken? re-download rock (verify size) + check backend grug re-test rock after every report: loads clean, zero spam at proper config.

how run

llama-server -m grug-27b-qat-Q4_K_M.gguf --mmproj mmproj-grug-27b-f16.gguf \
  -c 16384 --temp 0.6 --top-p 0.95 --top-k 20

need recent llama.cpp (qwen3_5 arch). grug think live in <think>, short on purpose. tool call use XML format. main model card: grug-27b.