grug-v1.1-qwen-3.8-27b-mtp — GGUF
grug in box, with draft head still inside. same weight as ProCreations/grug-v1.1-qwen-3.8-27b-mtp.
normal grug GGUF throw the MTP head away (--no-mtp) because it just make file
bigger for nothing. this build keep it, as blk.64.nextn.* tensor, so engine
that do speculative decoding can guess ahead.
| file | size | vs no-mtp build |
|---|---|---|
grug-27b-v1.1-mtp-Q8_0.gguf |
29.0 GB | +0.4 GB |
grug-27b-v1.1-mtp-Q6_K.gguf |
22.4 GB | +0.3 GB |
grug-27b-v1.1-mtp-Q4_K_M.gguf |
16.8 GB | +0.3 GB |
mmproj-grug-27b-v1.1-mtp-f16.gguf |
0.9 GB | eyes, same |
the small extra size IS the draft head.
which repo you want
- just run grug -> grug-v1.1-qwen-3.8-27b-gguf. smaller, 5 quant, no dead weight.
- speculative decoding -> this one.
draft head only help when your engine actually use it. if your runtime ignore
nextn, you carry 0.3 GB for nothing — take the other repo.
why draft head retuned
Qwen3.8 native head agree with grug verifier 90.04% of time at t+2. grug retune it on grug own output and get 95.37% — disagree 1-in-21 instead of 1-in-10. higher agree = more draft token accepted = faster.
number is top-1 agreement with verifier, not measured wall-clock speedup. receipt and method on the MTP model card.
run
llama-cli -m grug-27b-v1.1-mtp-Q4_K_M.gguf -p "write a function that flattens a nested list"
verified output identical to non-MTP grug. draft head change speed, never answer.
quant load-tested with llama-bench before upload. use medium reasoning
effort — full table on the
main card.
apache-2.0.