GLM-5.3-Flash-NVFP4-sel3-MTP

A modified GLM-5.3-Flash in NVFP4, with the MTP layer retained so speculative decoding works out of the box.

Architecture

• Glm5NextForConditionalGeneration (glm5_next), multimodal (image-text-to-text)
• 45 decoder layers + 1 MTP layer, hidden size 4096, 288 routed experts
• 1,048,576 context
• compressed-tensors nvfp4-pack-quantized, group size 16

Serving (vLLM)

vllm serve OpenYourMind/GLM-5.3-Flash-NVFP4-sel3-MTP \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --tool-call-parser glm47 \
  --enable-auto-tool-choice \
  --reasoning-parser glm45 \
  --enable-prefix-caching \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Drop the --speculative-config line to run without speculative decoding.

Requires an NVFP4-capable GPU (Blackwell / sm100+) for the native FP4 path.

Limitations

• Alignment behaviour differs from the base model. Use accordingly.
• Quantization is NVFP4; expect small deviations from the BF16 base.

Attribution

Derived from zai-org/GLM-5.3-Flash (MIT).