Muse-Glimmer-30B — MLX q4-affine

A 4-bit affine MLX quantization of meta-models/Muse-Glimmer-30B — the text tower (a Gemma-family multimodal; the text model is served here) — for MTPLX on Apple Silicon. Pairs with a dflash block-diffusion speculative-decoding drafter (see the MTPLX PR linked below).

Architecture

Custom Gemma-3-like text tower (reference: llama.cpp src/models/muse-glimmer.cpp):

sigmoid gated attention (self_attn.gate_proj, before o_proj)
parameter-free QK-norm with qk_scale_factor folded onto Q
NoPE on the global/full-attention layers (RoPE θ=500000 only on sliding layers)
Gemma (1+w) sandwich norms; RMS-normed embeddings (not ×√hidden); SiLU SwiGLU; logit_scale + tanh softcap

52 layers · hidden 6656 · GQA 32/2 · head-dim 128 · vocab 202048 · sliding-window 2048 (pattern-4) · untied lm_head.

Quantization

q4-affine mirroring unsloth's UD-Q4_K_XL recipe: 4-bit gs32 body; 5-bit lm_head + attention-output layers 45–51; norms full-precision. 5.06 bpw / 16 GB.

Quality

Base-model completion, greedy, through the real MTPLX /v1/completions serve path (batch-decode, so indentation is correct):

benchmark	pass@1
HumanEval-164	47.6%
HumanEval+ (stricter)	43.3%
MBPP-378	68.8%
MBPP+ (stricter)	56.9%

Speculative decoding (dflash)

Paired with a bf16 dflash block-diffusion drafter through MTPLX's MTP route. Decode is token-exact vs greedy AR. Throughput × block size (K), Python code continuation, M5 Max:

workload	AR tok/s	best K	best tok/s	×AR
repetitive code (1024 ctx)	27.2	16	59.3	2.18×
short code prompt	28.1	6	43.3	1.54×
diverse code (1024 ctx)	27.4	4	28.6	1.04×

Speedup is acceptance-bound: ~2.2× on predictable/boilerplate code, ~break-even on genuinely diverse code (where the optimum shifts to small K). Full per-K matrix in the MTPLX PR.

Usage (MTPLX)

mtplx serve --model OpensourceWTF/Muse-Glimmer-30B-MLX-q4 \
  --stock-ar --no-mtp --unsafe-force-unverified --yes

For speculative decoding, use the dflash pair bundle (target/ + drafter/ + dflash_pair.json) described in the PR.

OpensourceWTF/Muse-Glimmer-30B-MLX-q4