Grug v1.1 Qwen3.8 27B — AWQ INT4 + trained MTP

vLLM-oriented quantization of ProCreations/grug-v1.1-qwen-3.8-27b-mtp. Activation-aware W4A16 asymmetric INT4 with group size 128, calibrated on 128 Ultrachat samples at 1024 tokens. The trained MTP head remains BF16. The vision tower, embeddings, LM head, and precision-sensitive GatedDeltaNet a/b gates also remain BF16.

The quantized backbone comes from ProCreations/grug-v1.1-qwen-3.8-27b-awq-int4 at df0e54f797a99d980608849be9331ae247920406. This is exact because every one of the 15 backbone BF16 shards in the normal and MTP source repositories has the same Xet object hash. The trained MTP shard comes from ProCreations/grug-v1.1-qwen-3.8-27b-mtp@f8d12d372177041c52751f4e2987ccc17fd4f3d9.

Serve with MTP

vllm serve ProCreations/grug-v1.1-qwen-3.8-27b-mtp-awq-int4 --max-model-len 32768 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'

The MTP shard is explicitly present in model.safetensors.index.json, and mtp.* is excluded from the packed quantization scheme so vLLM loads its BF16 weights instead of looking for weight_packed tensors.