Grug v1.1 Qwen3.8 27B — FP8 + trained MTP

vLLM-oriented quantization of ProCreations/grug-v1.1-qwen-3.8-27b-mtp. FP8 E4M3 with 128×128 block-scaled weights and dynamic activations. The trained MTP head remains BF16. The vision tower, embeddings, LM head, and precision-sensitive GatedDeltaNet a/b gates also remain BF16.

The quantized backbone comes from ProCreations/grug-v1.1-qwen-3.8-27b-fp8 at 28c126cc9bd71e6b29e9592ab1f10bc433ce5cae. This is exact because every one of the 15 backbone BF16 shards in the normal and MTP source repositories has the same Xet object hash. The trained MTP shard comes from ProCreations/grug-v1.1-qwen-3.8-27b-mtp@f8d12d372177041c52751f4e2987ccc17fd4f3d9.

Serve with MTP

vllm serve ProCreations/grug-v1.1-qwen-3.8-27b-mtp-fp8 --max-model-len 32768 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'

The MTP shard is explicitly present in model.safetensors.index.json, and mtp.* is excluded from the packed quantization scheme so vLLM loads its BF16 weights instead of looking for weight_packed tensors.