Muse Spark Open 816B-A42B · Atomic Dynamic GGUF

Atomic Dynamic (AD) quants of Meta's Muse Spark Open with our own importance matrix. A file goes up here only when it beats the other GGUF builds of the same size or larger, measured side by side on one stand.

---

Built with Muse Spark Open. The model is Meta's, under the Muse Model License and its Usage Policy, both included in this repo (LICENSE.md, USE_POLICY.md).

Pick a file

KL divergence is how far the quantized model's predictions drift from the original BF16 weights; lower is better, zero means identical. top-1 is how often it picks the same next token the original would have picked.

The sizes are bytes on disk; the KV cache and compute buffers come on top and grow with the context. On a Mac the GPU may use only part of the unified memory by default: for the 256 GB files raise the limit first (sudo sysctl iogpu.wired_limit_mb=...). We measured these files on NVIDIA GPUs and have not run them on a Mac.

We publish a size only where our file beats the other GGUF builds of the same size or larger.

How they compare

We downloaded the other publishers' files and measured them with the same reference logits, held-out text, context, chunk count and llama.cpp build as ours. Each of our files against the other file of the same size or larger:

Paired chunk by chunk over the 97 chunks of the held-out text, Unsloth's file drifts more than ours by:

• UD-Q2_K_XL against AD-Q2_K: 4.9% (95% interval 1.2% to 9.0%); ours matches the original's top token 0.20 points more often (0.04 to 0.36);
• UD-IQ2_XXS against AD-IQ2_XXS: 13.1% (10.2% to 16.4%); top token +0.70 points (0.52 to 0.87);
• UD-IQ1_M against AD-IQ1_M: 12.5% (10.0% to 15.3%); top token +0.66 points (0.49 to 0.83);
• UD-IQ1_S against AD-IQ1_S: 12.9% (10.3% to 15.8%); top token +0.91 points (0.70 to 1.12).

Also measured: ggml-org Q2_K (304.1 GB) 0.0963 / 90.08%.

Where Unsloth's files are better than anything of ours of the same size or smaller (measured the same way):

If you need one of those sizes, take Unsloth's file.

What is inside

Atomic Dynamic is how we decide which tensor gets which bits. The files use only standard llama.cpp quantization types; upstream llama.cpp with muse-spark support reads them.

• About 97% of the weights are the 256 routed experts (gate, up and down projections of 4096 x 4096 each, in the model's 4096-wide latent space). Only they are compressed hard.
• Attention (q, k, v, output, gate) and the latent projections stay at q8_0; the token embedding and the output head at q6_k.
• Bits move between experts and blocks without dropping anything into a lower precision regime. Each file starts from the expert types of the file it faces and changes only what costs almost nothing:
  • AD-Q2_K: gate and up q2_k, down q3_k on 38 blocks; on 8 blocks (0, 1, 56 to 61) gate and up iq3_xxs, down iq4_xs; on 16 blocks (29, 33 to 46, 52), the ones with the least activation energy into the expert down projections, gate and up iq2_xs, down iq3_xxs (an i-quant that nearly matches q2_k / q3_k with an importance matrix, at fewer bits);
  • AD-IQ2_XXS: every expert iq2_xxs, except the down projections of blocks 0, 1 and 56 to 61 at iq2_xs; paid for by the attention gate and the latent projections at q6_k;
  • AD-IQ1_M: gate and up iq1_m, down iq2_xxs (where UD-IQ1_M has iq1_s and iq2_s); blocks 0, 1 and 56 to 61 at iq2_xxs and iq2_xs;
  • AD-IQ1_S: every expert iq1_s, except blocks 0, 1 and 56 to 61 at iq1_m; paid for by the attention gate and the latent projections at q6_k and attention q and output at q5_k.
• Attention and the latent projections stay at q8_0 or q6_k, the token embedding at q4_k or q6_k, the output head at q6_k (exact types per file in the --tensor-type rules below and in the file itself).
• Every row length in this model is a multiple of 256, so no quantization type is swapped silently; we checked the type of every tensor in the finished file.

Running it

llama-server -m Muse-Spark-Open-816B-A42B-AD-IQ1_M-00001-of-00005.gguf -ngl 99 -c 16384 --jinja

Point llama.cpp at the first part of a file; it loads the rest. The chat template is inside the file. The recommended sampling (temperature 1.0, top-p 1.0, top-k 64) is stored in the file's metadata. The model always reasons; the depth is set with reasoning_effort (low, medium, high, xhigh, the default), for example --chat-template-kwargs '{"reasoning_effort": "medium"}'.

Needs llama.cpp with muse-spark support: commit ba3c05f71da8 (the merge of ggml-org/llama.cpp#30616) or later. We built and measured everything with that commit.

Images and audio

The vision and audio encoders are in a separate file, shared by every quant:

• mmproj-Muse-Spark-Open-816B-A42B-F16.gguf for most people
• mmproj-Muse-Spark-Open-816B-A42B-BF16.gguf if your setup prefers bfloat16
• mmproj-Muse-Spark-Open-816B-A42B-Q8_0.gguf, smaller

llama-mtmd-cli -m Muse-Spark-Open-816B-A42B-AD-Q2_K-00001-of-00007.gguf \
  --mmproj mmproj-Muse-Spark-Open-816B-A42B-F16.gguf \
  --image your-photo.jpg -ngl 99 -c 16384 -p "What is in this image?"

These are ggml-org's conversions of Meta's encoders, copied here unchanged. We have not tested image or audio input with AD-Q2_K.

How we measured

• Reference: the BF16 weights in GGUF, run unquantized (ggml-org's conversion of Meta's checkpoint with llama.cpp's converter). Perplexity on the held-out text is 4.2948 plus or minus 0.0275.
• Held-out text: eval_neutral from our calibration dataset (30 languages), never used for calibration. 97 chunks at 4096 context.
• Metric: per-token KL divergence of each file's predictions against the reference, plus top-1 agreement.
• Hardware: 8x A100-SXM4 80 GB for every file we publish and every file we compare them with. AD-IQ2_M in the last table was measured on 8x A100-SXM4 40 GB; UD-IQ2_S and UD-IQ2_XXS, measured on both machines, gave the same numbers to six digits.
• llama.cpp ba3c05f71da8, the same build for every file in the tables.

How the importance matrix was made

The matrix comes from 2.95M tokens of our calibration corpus (agentic and tool use, code, reasoning, many languages, long documents, and a sweep that reaches 99.99% of the vocabulary), every conversation rendered through this model's own chat template and read with --parse-special:

• 2.52M tokens at context 512, split over six workers;
• 0.43M tokens of 16k to 32k token documents at context 32768.

It was collected on a Q8_0 copy of the model, not on the BF16 weights: the BF16 is 1.64 TB, more than the 640 GB of VRAM of the machines we had. Q8_0 activations are very close to the BF16 ones, but this is a proxy, and we say so.

To rebuild AD-Q2_K with your own matrix, the quantization rules are:

llama-quantize --imatrix imatrix.gguf \
  --tensor-type '^token_embd\.weight$=q6_k' \
  --tensor-type '^output\.weight$=q6_k' \
  --tensor-type '^blk\.(0|1|56|57|58|59|60|61)\.ffn_down_exps\.weight$=iq4_xs' \
  --tensor-type '^blk\.(0|1|56|57|58|59|60|61)\.ffn_(gate|up)_exps\.weight$=iq3_xxs' \
  --tensor-type '^blk\.(29|33|34|35|36|37|38|39|40|41|42|43|44|45|46|52)\.ffn_down_exps\.weight$=iq3_xxs' \
  --tensor-type '^blk\.(29|33|34|35|36|37|38|39|40|41|42|43|44|45|46|52)\.ffn_(gate|up)_exps\.weight$=iq2_xs' \
  --tensor-type '^blk\.\d+\.ffn_down_exps\.weight$=q3_k' \
  --tensor-type '^blk\.\d+\.ffn_(gate|up)_exps\.weight$=q2_k' \
  --tensor-type '^blk\.\d+\.ffn_latent_(down|up)\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.attn_(q|k|v|output|gate)\.weight$=q8_0' \
  Muse-Spark-Open-816B-A42B-BF16-00001-of-00022.gguf Muse-Spark-Open-816B-A42B-AD-Q2_K.gguf Q2_K

For AD-IQ1_M:

llama-quantize --imatrix imatrix.gguf \
  --tensor-type '^token_embd\.weight$=q4_k' \
  --tensor-type '^output\.weight$=q6_k' \
  --tensor-type '^blk\.(0|1|56|57|58|59|60|61)\.ffn_down_exps\.weight$=iq2_xs' \
  --tensor-type '^blk\.(0|1|56|57|58|59|60|61)\.ffn_(gate|up)_exps\.weight$=iq2_xxs' \
  --tensor-type '^blk\.\d+\.ffn_down_exps\.weight$=iq2_xxs' \
  --tensor-type '^blk\.\d+\.ffn_(gate|up)_exps\.weight$=iq1_m' \
  --tensor-type '^blk\.\d+\.ffn_latent_(down|up)\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.attn_(k|v|gate)\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.attn_(q|output)\.weight$=q6_k' \
  Muse-Spark-Open-816B-A42B-BF16-00001-of-00022.gguf Muse-Spark-Open-816B-A42B-AD-IQ1_M.gguf IQ1_M

For AD-IQ2_XXS: every expert iq2_xxs except ^blk\.(0|1|56|57|58|59|60|61)\.ffn_down_exps\.weight$=iq2_xs; attention q, output, gate and the latent projections q6_k, attention k and v q8_0, token_embd q4_k, output q6_k. For AD-IQ1_S: every expert iq1_s except ^blk\.(0|1|56|57|58|59|60|61)\.ffn_(gate|up|down)_exps\.weight$=iq1_m; attention q and output q5_k, attention gate and the latent projections q6_k, attention k and v q8_0, token_embd q4_k, output q6_k. The first rule that matches a tensor wins.

Model details

Mixture of experts, 816B parameters in total and about 42B active per token: 62 layers, hidden size 8192, grouped-query attention (128 query heads, 16 key-value heads, head size 64) with sliding-window attention in three of every four layers, 256 experts with 8 active per token in a 4096-wide latent space, vocabulary 202,048. Meta's config sets a maximum context of 1,048,576 tokens; we measured at 4096 and did not test long context.