AxionML Muse-Glimmer-30B-NVFP4

Mirrored by AxionML for open-source serving and deployment use cases. Part of AxionML's effort to provide ready-to-serve quantized models for the community.

Quantized by NVIDIA. The weights in this repository are an unmodified copy of nvidia/Muse-Glimmer-30B-NVFP4 (revision 47818374517751c48c55cde2621594926b1888b6). All credit for the quantization belongs to NVIDIA.

This is an NVFP4-quantized version of meta-models/Muse-Glimmer-30B (29.6B parameters including the perception encoder), quantized with NVIDIA Model Optimizer.

About NVFP4 quantization: NVFP4 on Blackwell couples a compact E2M1 FP4 codebook with blockwise FP8 (E4M3) scaling over 16-element micro-blocks, so that 4-bit stored values remain numerically useful for neural-network computation. The E2M1 codebook provides a small, nonuniform set of representable magnitudes up to ±6 and relies on saturating behavior rather than IEEE NaN/Inf encodings to maximize usable range per bit. Using an FP8 block scale (rather than power-of-two-only E8M0) enables fractional scales and error-minimizing scale selection. On Blackwell Tensor Cores, native FP4 multipliers exploit E2M1 simplicity while higher-precision FP32 accumulation protects dot-product accuracy.

Ready for commercial and non-commercial use under Apache 2.0.

Model Summary


Architecture	Dense causal transformer with perception encoder (`MuseGlimmerForConditionalGeneration`)
Parameters	29.6B (including vision encoder)
Input	Interleaved text and images (up to 4,096 visual tokens per image)
Context Length	131K tokens
Reasoning	Controllable strength: low / medium / high / xhigh
Checkpoint Size	~24.7 GB (vs ~60 GB BF16)

Evaluation Results

Benchmark	BF16	NVFP4 (mixed)
Terminal-Bench 2.1 pass@1	45.22 ± 0.92	47.05 ± 1.42
GDPVal normalized Elo	0.20282	0.23798
GPQA Diamond pass@1	83.81 ± 0.33	83.02 ± 0.40
MMMU-Pro	74.22	73.58
SciCode subtask accuracy	47.63	49.70
IFBench strict pass@1	76.58 ± 0.53	78.74 ± 0.52
AA-LCR mean reward	76.25	75.56

Scores reported by NVIDIA for this checkpoint: high reasoning, temperature=1.0, top_p=0.95, top_k=64. Gains over BF16 are within sampling noise.

Quantization Details

Quantization format: mixed precision via ModelOpt AutoQuantize — each MLP, self-attention projection and lm_head layer assigned W4A16 NVFP4, FP8, or BF16 under a 5.5 effective-bit budget (6.6 effective bits overall)
Vision encoder: BF16, not quantized
KV cache: unquantized (auto at serving time)
Calibration dataset: 512 × 2,048-token sequences from Nemotron-Post-Training-Dataset-v3
Tool: NVIDIA Model Optimizer v0.46.0 (recipe)

Usage

Deploy with SGLang

sglang serve \
    --model-path AxionML/Muse-Glimmer-30B-NVFP4 \
    --language-model-only \
    --reasoning-parser muse \
    --tool-call-parser muse \
    --kv-cache-dtype fp8_e4m3 \
    --mem-fraction-static 0.85 \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path meta-models/Muse-Glimmer-30B-assistant \
    --speculative-draft-model-quantization fp8

Deploy with vLLM

vllm serve AxionML/Muse-Glimmer-30B-NVFP4 \
    --trust-remote-code \
    --enable-prefix-caching \
    --enable-chunked-prefill \
    --enable-auto-tool-choice \
    --tool-call-parser muse_glimmer \
    --reasoning-parser muse_glimmer \
    --kv-cache-dtype auto \
    --max-model-len 131072 \
    --mamba-cache-mode align

Validated upstream on one B200 (SGLang, text-only with DFlash) and with vLLM 0.28.0. For matched quality use temperature=1.0, top_p=0.95, top_k=64, thinking enabled. See USAGE_POLICY.md.

Limitations

The base model was trained on data that may contain toxic language and societal biases. The quantized model inherits these limitations. It may generate inaccurate, biased, or offensive content. Please refer to the original model card and the upstream quantized model card for full details.

Credits

Base model: meta-models/Muse-Glimmer-30B
Quantization: nvidia/Muse-Glimmer-30B-NVFP4 by NVIDIA
Mirror: AxionML