grug 27b v2 — gguf

grug brain squeezed into smaller rock. draft head come with rock now.

these quantizations of ProCreations/grug-27b-v2. every text GGUF carry Grug-tuned native MTP head. no separate draft model to fetch. optional mmproj file is preserved vision projector — eye rock, not draft brain.

pick your rock

grug test squeezed brain too

rocks converted and tested on HF Jobs with llama.cpp commit 2a3005c23f60cb38dab70b8ea2ddbd969bcf3e87. every quant load on H200, MTP off and on. grug check generation, titles, tools, history handling, executable code.

results include 16-case coding smoke test for each quant. small smoke test, not full quantized benchmark. grug not claim rock pass hunt it never take.

how wake rock

llama-server \
  -hf ProCreations/grug-27b-v2-gguf:Q4_K_M \
  -ngl 99 -c 12288 --jinja \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  --repeat-penalty 1.05 --repeat-last-n 12288 \
  --spec-type draft-mtp --spec-draft-n-max 2 -ngld 99

pick low, medium, xhigh through chat endpoint's reasoning_effort. default medium. bigger effort give grug more thought, not guaranteed better score. hard thought sometimes leave Grug style. some loops still live. main model card and full results show bruise too.

keep old thought in right pouch

pinned llama-server accept canonical reasoning_content and inline history. it drop reasoning input alias before template render. grug record this as one failed capability probe in each mode, not sweep under cave rug.

forwarding vLLM-style histories? use normalize_messages.py. usage.md explain plumbing.

what rock not promise

MTP head packed by default. speculative decoding still need runtime switch. MTP paths not guaranteed identical greedy tokens, speed gain not universal.

vision weights preserved, but no new vision benchmark. full configured-context quality not established. grug keep eye; grug not invent eye score.

rock receipts

gguf_conversion.json record files and embedded head counts. check downloads against SHA256SUMS. all model compute happen on HF Jobs; HF Jobs ledger cover whole main-model and GGUF build. Apache-2.0 license.

grug made by ProCreations. base brain by Qwen team.