grug-35b-v2-gguf
35b brother reborn, squeezed into rocks. MoE brain: fast like small model,
smart like big model. grug think dense inside <think>, answer normal english.
full story + all benchmark: grug-35b-v2.
rocks
| file | quant |
|---|---|
| grug-35b-v2-Q8_0.gguf | Q8_0 |
| grug-35b-v2-Q6_K.gguf | Q6_K |
| grug-35b-v2-Q5_K_M.gguf | Q5_K_M |
| grug-35b-v2-Q4_K_M.gguf | Q4_K_M - grug pick this |
| grug-35b-v2-Q3_K_M.gguf | Q3_K_M |
| mmproj-grug-35b-v2-f16.gguf | eye rock (vision) |
every rock load-tested before upload. old grug-35b repeat-word sickness: extinct in v2 (0 loops in every gauntlet, 100% think closed).
if rock act broken
single-token spam ("/" forever etc) = NOT the rock. hybrid DeltaNet brain CANNOT survive llama.cpp context-shift: old builds shift on context overflow and corrupt the recurrent state into token spam. fix:
- use RECENT llama.cpp (qwen3_5 support; new builds refuse instead of shift)
- agent frontends (OpenCode etc): set
-c 16384or bigger - still broken? re-download rock (verify size) + check backend grug re-test rock after every report: loads clean, zero spam at proper config.
how run
llama-server -m grug-35b-v2-Q4_K_M.gguf --mmproj mmproj-grug-35b-v2-f16.gguf \
-c 16384 --temp 0.6 --top-p 0.95 --top-k 20
need recent llama.cpp (qwen3_5_moe arch). MTP not included. grug made by ProCreations. base brain by DeepReinforce.