Muse-Glimmer-30B GPTQ 4-bit

GPTQ 4-bit (W4, group size 32) quantization of meta-models/Muse-Glimmer-30B (revision a4e59da52a7bc87ae7251dd5545c0dd437c44b68), a 29.8B dense text + vision model. The 52-layer text decoder is quantized to INT4; the vision tower, vision adapter/projection, embeddings, norms and LM head stay in BF16. The checkpoint is 23 GB (BF16 original: 59.6 GB).

Quantization Details

Component	Precision	Notes
`self_attn.{q,k,v,gate,o}_proj`	INT4 GPTQ	all 52 decoder layers (incl. the attention output gate)
`mlp.{gate,up,down}_proj`	INT4 GPTQ	all 52 decoder layers
Vision tower, vision adapter, vision projection	BF16	unchanged from the original
Embeddings, LM head, norms	BF16

416 quantized modules in total.

GPTQ configuration:

Bits: 4
Group size: 32
Symmetric: Yes
desc_act: No
true_sequential: Yes
mse: 2.0
damp_percent: 0.05
Fallback: RTN at a 0.5% calibration-coverage threshold

Calibration

Dataset: mixed evol-codealpaca-v1 (code) + C4 (general English), the same set used for our other GPTQ releases
Samples: 2048, binned uniformly across context lengths 256–2048 tokens, each passed as a single user turn through the model's chat template
Quantizer: GPTQModel v7.3.5 (muse_glimmer model definition)

The exact quantization script is included as quantize.py.

Verification

Served with vLLM (tensor parallel 4, --tool-call-parser muse_glimmer --reasoning-parser muse_glimmer):

Text: correct, coherent answers (e.g. 17 × 23 → 391)
Image: read a newspaper front page correctly (masthead, printed date, lead headline)
Tool calling: a weather tool request returned a parsed get_weather call with the right arguments
No server faults

Perplexity and fidelity against the BF16 original have not been measured yet.

Usage

vLLM

vllm serve btbtyler09/Muse-Glimmer-30B-GPTQ-4bit \
  --tensor-parallel-size 4 \
  --max-model-len 65536 \
  --enable-auto-tool-choice --tool-call-parser muse_glimmer --reasoning-parser muse_glimmer

Reasoning strength

The chat template has no on/off thinking switch. It uses Reasoning strength: low | medium | high | xhigh (default high), set in the system prompt or via chat_template_kwargs={"reasoning_strength": "low"}. The model's reasoning channel usually opens by restating the user's request; this matches the BF16 original and is not caused by quantization.

License

Apache-2.0, inherited from the base model. The base model's USAGE_POLICY.md is included and applies to this checkpoint.

btbtyler09/Muse-Glimmer-30B-GPTQ-4bit