Muse-Glimmer-30B — MLX MXFP4

MLX MXFP4 (4-bit microscaling float) quantization of meta-models/Muse-Glimmer-30B, a ~30B dense causal transformer with a ~1.8B perception encoder, built for autonomous agentic tasks on consumer hardware. Runs on Apple Silicon via mlx-vlm. Stays image-text-to-text — the vision tower and projector are kept in bf16.


Precision	MXFP4 (E2M1 + E8M0 shared scale, group size 32)
Bits per weight	5.009 bpw
On-disk size	18.6 GB
Quantized	language model (incl. `lm_head`)
Kept in bf16	vision tower + vision adapter/projection
Fits in	24 GB unified memory (with room for KV cache)
Measured speed	~16 tok/s (Apple M-series, greedy)

This is the variant to use on a 24 GB Mac. For 32 GB+ machines, the higher-fidelity MXFP8 build is available.

Verification

Quantized with mlx_lm.quantize_model (mode mxfp4, group 32), keeping the vision path in bf16. Smoke-tested with deterministic greedy decoding on arithmetic prompts with known answers (the chat template's reasoning channel is allowed to run before the final answer):

Prompt	Expected	MXFP4 answer
`84 * 3 / 2`	126	"84 × 3 = 252 · 252 ÷ 2 = 126"	✅
`17 + 28`	45	45	✅
`256 / 4`	64	64	✅
`13 * 12`	156	"13×10=130, 13×2=26, 130+26=156"	✅
`1000 - 333`	667	667	✅
square of 15	225	"15 × 15 = 225"	✅

6/6 correct with coherent step-by-step reasoning. The MLX implementation correctly handles this architecture's non-standard pieces (per-layer NoPE on the full-attention layers, final_logit_softcapping, qk_scale_factor, output_multiplier, gated attention, centered RMSNorm).

Usage (mlx-vlm)

pip install -U mlx-vlm   # needs >= 0.6.12 for the muse_glimmer architecture

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("sahilchachra/Muse-Glimmer-30B-MXFP4")
config = model.config

messages = [{"role": "user", "content": "What is 84 * 3 / 2?"}]
prompt = apply_chat_template(processor, config, messages, add_generation_prompt=True)
text = generate(model, processor, prompt, max_tokens=256, verbose=True)

For image input, pass an image to apply_chat_template / generate per the mlx-vlm docs — the vision path is preserved in bf16.

Recommended sampling (from the base model card): temperature=1.0, top_p=0.95, top_k=64. Reasoning strength is set via the system prompt (Reasoning strength: low|medium|high|xhigh).

Notes & limitations

tie_word_embeddings=false, so the 1.35B-param lm_head is a separate matrix and is quantized at 4-bit — the part most sensitive to numeric precision. Arithmetic was verified correct above, but for threshold-sensitive or heavy numeric/agentic workloads the MXFP8 build has more headroom.
Inherits all capabilities and limitations of the base model. See the original model card and usage policy.