grug-9b-Q5_K_L-imat-GGUF
GGUF quantization of ProCreations/grug-9b to Q5_K_L with importance matrices. Uses Q8_0 for token embedding and output weights, and Q5_K_M config for all other weights.
Should have a bit higher accuracy than regular Q5_K_M, to help with more complex/long contexts and accuracy in edge cases.
Be warned, the calibration file is mostly English text and code, so performance writing in Chinese may suffer.
Original Model Card Below
grug-9b
grug take big-brain bird model. grug teach bird think small. bird still code good. bird now think 36 token where old bird think 570. this whole model, LoRA already squished inside. download, run, done.
base bird: Ornith-1.0-9B (dense 9B, MIT, very smart, very chatty inside head) grug medicine: LoRA r=64 on grug-think dataset — 100k agent trajectory where all thinking rewritten short grug style. one full epoch, ~140M token, one RTX PRO 6000, ~13 hour.
numbers. grug measure, not guess
same harness both birds. same prompt. greedy. no cherry pick. eval code live in dataset repo (scripts/eval_grug.py).
how good
| test | old bird | grug bird | change |
|---|---|---|---|
| HumanEval pass@1 | 91.5% | 79.3% | -12.2 |
| MBPP pass@1 | 80.0% | 78.0% | -2.0 |
| agent replay: tool call valid | 100% | 88.9% | -11.1 |
| agent replay: RIGHT tool picked | 77.8% | 88.9% | +11.1 |
how cheap
| test | old bird tokens (think part) | grug bird tokens (think part) | change |
|---|---|---|---|
| HumanEval | 958 (570 think) | 249 (36 think) | -74% (-94% think) |
| MBPP | 594 (305 think) | 148 (38 think) | -75% (-88% think) |
| agent replay | 137 (99 think) | 73 (14 think) | -47% (-85% think) |
how fast
whole benchmark suite, same GPU, same everything:
| old bird | grug bird | |
|---|---|---|
| wall time | 87.4 min | 26.1 min |
3.3x faster. tokens = time = money. grug bird leave money in pocket.
per SOLVED problem old bird spend ~1047 token. grug bird spend ~314. three grug answer for price of one old answer.
real grug bird think trace
<think>Tuple field likely in fields.py. View file to find implementation.</think>
<think>Need exact line numbers for Tuple class. Use grep.</think>
<think>Understand how _bind_to_schema works in base Field class. Tuple likely
missing override that binds inner fields.</think>
real hypothesis chain. no "Let me carefully analyze". grug already analyzing. grug just do.
honest part. grug always honest
- HumanEval drop 12 point. on hardest problem, big thinking was doing real work. grug bird trade some hard-problem muscle for 4x cheaper everything. if that trade bad for your cave, use old bird there
- MBPP basically same. agent tool-choosing BETTER (+11). grug style maybe help bird pick right club
- tool-call format valid 100 -> 88.9. sometimes grug bird talk when should poke tool
- fix ideas grug not try yet: mix 10-20% long-think example in data, or second epoch
how use
from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch
model = AutoModelForImageTextToText.from_pretrained(
"ProCreations/grug-9b", dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("ProCreations/grug-9b")
messages = [{"role": "user", "content": "Write a function that checks if a number is prime."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
need transformers >= 5.8 (qwen3_5 architecture). thinking come in <think> tag. tool calling use same XML <tool_call><function=...> format as base bird. vision tower still inside, untouched, frozen during tune.
family
- dataset: ProCreations/grug-think — 100,891 example, think median 11 word
- adapter only (small download, need base): ProCreations/grug-ornith-9b-lora
grug done. model think small. work same-ish. much cheap. go build.