Muse-Spark-Open-816B-A42B — leCore mixed-precision MLX (260 GB)

Built with Muse Spark Open.

A mixed-precision 2/3/4-bit MLX quantization of the text tower of meta-models/Muse-Spark-Open-816B-A42B (816B total / 42B active MoE). Bit-widths are chosen per tensor by leCore: an exact-KL sensitivity pass on the full-precision model (every tensor tried at 2, 3 and 4 bits, its effect on the final logits measured), followed by a knapsack allocation to a size budget.

Size

• Average 2.54 bits per weight overall (all stored bytes incl. scales/biases ÷ 820B stored parameters); quantized linear/expert weights average 2.02 bits (group size 64, affine MLX quantization), tensor histogram {2-bit: 204, 3-bit: 162, 4-bit: 192}; embeddings 4-bit, LM head 6-bit; routers and norms kept in BF16.
• The size target was set to 260 GB; that is close to the floor for this group size (an all-2-bit model would be about 257 GB).
• Vision/audio towers are not included (text-only).

Measured quality (vs the original BF16 model)

Evaluated with the same streamed engine and identical tokens for both models (BF16 weights vs the exact leCore-quantized weights, bit-identical to MLX's quantizer).

The MMLU subset is small (228 questions, about ±2-3 points of sampling noise), so treat it as a sanity check, not a full benchmark. Not measured: full MMLU, GSM8K, long-context, multimodal, agentic/tool-use benchmarks, and a stock uniform-quantization baseline. Perplexities are on short 512-token windows, so absolute values are not comparable to other published numbers; the BF16 vs leCore delta is the meaningful part.

Hardware requirements

The weights alone are 260 GB, so this needs roughly 273+ GB of unified/GPU memory plus KV cache. It does not fit a single 128 GB machine (or a 192 GB one); think a 512 GB Mac Studio-class machine or a multi-GPU MLX setup. (No GGUF: llama.cpp has no muse_spark architecture yet.)

Files in this repo

Usage (MLX)

mlx-lm does not ship a muse_spark model yet. Copy muse_spark.py from this repo into mlx_lm/models/ (tested with mlx-lm 0.32.1, mlx 0.32.4), then:

from mlx_lm import load, generate
model, tokenizer = load("staccs/Muse-Spark-Open-816B-A42B-lecore-mlx")
print(generate(model, tokenizer, prompt="Explain mixture-of-experts in two sentences.", max_tokens=128))

License and gating

This is a Derivative of Muse Spark Open and is distributed under the Muse Model License (version: Muse Spark Open, Copyright (c) 2026 Meta Platforms, Inc.), included here as LICENSE.md, together with the Usage Policy (USE_POLICY.md, Exhibit A of the license), as the license requires for all copies and Derivatives. Key terms (read the full text): you must comply with the Usage Policy; you must prominently display "Built with Muse Spark Open" in any product/service (or its documentation) that uses this model; and Model-as-a-Service providers whose affiliates exceed USD 20M gross revenue in any 12 months must notify Meta and sign a separate agreement first.

The base repository is gated (access requires accepting the Muse Model License and Meta's terms on the base model page). By downloading or using these weights you agree to the same license and Usage Policy.

Base model: meta-models/Muse-Spark-Open-816B-A42B — see its model card for intended use, limitations and safety information.