Gemma 4 12B QAT MTP Assistant GGUF

GGUF quantizations converted from google/gemma-4-12B-it-qat-q4_0-unquantized-assistant.

Tested with llama.cpp b9549+ (Gemma 4 MTP support).

Recommendations

• Q4_0_Q4emb — recommended for most users
• Q8_0 — highest precision

Files

• gemma-4-12b-qat-it-assistant-Q4_0-Q6emb.gguf
• gemma-4-12b-qat-it-assistant-Q4_0_Q4emb.gguf
• gemma-4-12b-qat-it-assistant-Q8_0.gguf

Notes

Q4_0_Q4emb keeps token_embd.weight in Q4_0 rather than the higher-precision embedding quantization normally used by llama.cpp.

Initial testing showed similar draft acceptance rates while reducing file size.

Example

llama-server \
  -m gemma-4-12B-it-qat-Q4_0.gguf \
  -md gemma-4-12b-qat-it-assistant-Q4_0_Q4emb.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 2