The capacity / size-optimal tier of the Inkling · Alis MLX Dynamic family — siblings: the ~3.71 bpw quality build (single 512 GB box, golden spot) and the ~6.6 bpw two-box Q6 performance build. Full family: Inkling 975B · Alis MLX Dynamic collection.
Apple Silicon (MLX) mixed-precision quantization of thinkingmachines/Inkling — a 975B-class multimodal Mixture-of-Experts model (66 hybrid decoder layers, 256 routed experts (top-6) + 2 shared per MoE layer, hidden 6144, sliding-window attention + short-convolution hybrid, vision + audio front-ends, 201K vocab).
This build answers a different question than its quality sibling: how small can 975B get while every design-critical path keeps real precision? ~2.72 bits/weight, 299 GiB on disk — a 975B-class multimodal MoE that fits a 512 GB Mac with ~150 GiB to spare for KV cache and activations, with quantization scales certified by the same fully receipt-sealed, layer-local ALIS-DWQ pass as the quality tier.
A 2-bit floor for the expert bulk, with targeted promotions exactly where damage concentrates: late layers, early down-projections, shared experts, and every control path. Nothing here is uniform-2-bit — the recipe spends its bits dynamically.
| Component | Precision | Why |
|---|---|---|
Routed experts w13/w2, layers 3–55 |
2-bit affine, g128 | the parameter bulk at the floor |
| Routed experts, layers 56–65 | 3-bit, g128 | late layers shape the output distribution |
Early down-proj w2, layers 3–7 |
3-bit, g128 | down-projections compound early errors |
| Shared experts (all layers) | 4-bit, g64 | on every token, fine-grained groups |
| Attention (all 5 proj., hybrid + sconv) | 6-bit, g128 | every token's critical path, kept high |
| Dense MLP (layers 0–1) | 6-bit, g128 | early-layer protection |
| Embeddings · LM head | 8-bit, g128 | distribution-sensitive |
| Layer 2 (entirely) · router/gates · norms | BF16 | control flow stays exact |
| Vision + audio towers | BF16 | modality front-ends kept exact |
MTP head dropped. The base model's multi-token-prediction head (
model.mtp.*, 5.26B params) is a speculative-decoding accessory the MLX runtime does not use; it is dropped at conversion. That is why all three tiers report 947.1B logical params from a 952.4B source — the tiers quantize an identical parameter set.
| Base model | thinkingmachines/Inkling (975B-class; 952.4B source / 947.1B logical params) |
| Bits/weight | 2.72 effective (321.4 GB total) |
| On-disk size | 321.4 GB (299.3 GiB), 108 shards |
| Format | MLX safetensors (inkling_mm_model) |
| Modalities | text + image + audio inputs |
| Attention | sliding-window (512) hybrid + short conv — KV stays small at long context |
Most public quants ship converted weights. This one ships converted weights plus a machine-checkable certification trail. The quantization scales/biases were passed through layer-local ALIS-DWQ (alis-dwq) against exact BF16 teacher activations, under a guard-and-receipt harness built for reproducible two-box runs:
advanced-completion receipt (status: pass), and the merged shards were rebuilt with per-tensor byte verification (patched tensors match their sealed patch payloads; all others match baseline exactly).The result is conservative by construction: provably-not-worse-than-baseline weights with certified improvements where they were found — not a quant that traded unknown regressions for a benchmark bump. (A higher-lr DWQ tier on the same evidence rails is planned as a separate revision.)
| Build | bpw | Size | For |
|---|---|---|---|
| quality / golden spot | 3.71 | 409 GiB | single 512 GB Mac, best quality in one box |
| this — capacity / size-optimal | 2.72 | 299 GiB | 512 GB Mac with generous headroom; 320–384 GB boxes |
| Q6 teacher (two-box) | 6.60 | 728 GiB | maximum fidelity, 2 × 512 GB pipeline serving |
The inkling_mm_model architecture is served by the project's MLX port (hybrid sliding-window + short-conv attention, 256-expert gather MoE, multimodal towers); upstream mlx-lm support is not yet merged. Until the port lands upstream, treat this repo as weights + provenance, loadable with the Inkling MLX runtime from the alis pipeline:
# Inkling MLX runtime (alis pipeline port)
from inkling_mlx.model import load_model
model = load_model("avlp12/Inkling-975B-Alis-MLX-Dynamic-2.7bpw")
Sliding-window attention keeps the KV cache small, so long contexts are memory-cheap relative to dense-attention peers. At 299 GiB of weights, a 512 GB box runs this with ~150 GiB of true headroom — and 384 GB machines fit it comfortably.
Built for Apple Silicon at 320 GB+ unified memory (M2/M3 Ultra class). On 512 GB machines, consider the 3.7 bpw quality build for maximum single-box quality.
The DWQ schedule is complete and certified (66/66 sealed layer receipts + advanced-completion pass; 7 accepted improvements, 59 certified-neutral).
Runtime verification (measured, this exact artifact):
load_weights(strict=True) — no missing/unexpected tensors.The remaining project gates — two-box candidate held-out evaluation against the Q6 teacher and final quality validation — are in progress; this card will be updated with those measurements as they land. Until then, treat quality claims as certified-not-worse than the baseline conversion rather than benchmarked. At 2.72 bpw, expect visible quality loss versus the 3.7 bpw tier on hard tasks — this tier trades fidelity for footprint by design.
ml-explore.inkling_mm_model port: Alis (avlp12).Alis (avlp12) (2026). Inkling-975B-Alis-MLX-Dynamic-2.7bpw — 2.7 bpw certified layer-local DWQ MLX quantization of Inkling. https://huggingface.co/avlp12/Inkling-975B-Alis-MLX-Dynamic-2.7bpw
model-00107-of-00108.safetensors 19 GBmodel-00001-of-00108.safetensors 12 GBmodel-00037-of-00108.safetensors 4.2 GBmodel-00072-of-00108.safetensors 4.1 GBmodel-00104-of-00108.safetensors 4.1 GBmodel-00021-of-00108.safetensors 4.1 GBmodel-00003-of-00108.safetensors 4.1 GBmodel-00016-of-00108.safetensors 4.0 GBmodel-00062-of-00108.safetensors 4.0 GBmodel-00018-of-00108.safetensors 4.0 GBmodel-00006-of-00108.safetensors 4.0 GBmodel-00010-of-00108.safetensors 4.0 GBmodel-00074-of-00108.safetensors 3.9 GBmodel-00049-of-00108.safetensors 3.6 GBmodel-00036-of-00108.safetensors 3.6 GBmodel-00045-of-00108.safetensors 3.5 GBmodel-00044-of-00108.safetensors 3.5 GBmodel-00042-of-00108.safetensors 3.5 GBmodel-00048-of-00108.safetensors 3.4 GBmodel-00098-of-00108.safetensors 3.4 GBmodel-00041-of-00108.safetensors 3.4 GBmodel-00064-of-00108.safetensors 3.4 GBmodel-00026-of-00108.safetensors 3.0 GBmodel-00059-of-00108.safetensors 3.0 GBmodel-00011-of-00108.safetensors 3.0 GBmodel-00092-of-00108.safetensors 3.0 GBmodel-00069-of-00108.safetensors 2.9 GBmodel-00105-of-00108.safetensors 2.9 GBmodel-00032-of-00108.safetensors 2.9 GBmodel-00076-of-00108.safetensors 2.9 GBmodel-00080-of-00108.safetensors 2.9 GBmodel-00094-of-00108.safetensors 2.9 GBmodel-00040-of-00108.safetensors 2.9 GBmodel-00087-of-00108.safetensors 2.9 GBmodel-00025-of-00108.safetensors 2.9 GBmodel-00082-of-00108.safetensors 2.9 GBmodel-00028-of-00108.safetensors 2.8 GBmodel-00020-of-00108.safetensors 2.8 GBmodel-00023-of-00108.safetensors 2.8 GBmodel-00070-of-00108.safetensors 2.8 GBmodel-00053-of-00108.safetensors 2.8 GBmodel-00058-of-00108.safetensors 2.8 GBmodel-00079-of-00108.safetensors 2.8 GBmodel-00029-of-00108.safetensors 2.8 GBmodel-00078-of-00108.safetensors 2.8 GBmodel-00096-of-00108.safetensors 2.8 GBmodel-00043-of-00108.safetensors 2.8 GBmodel-00073-of-00108.safetensors 2.8 GBmodel-00067-of-00108.safetensors 2.8 GBmodel-00099-of-00108.safetensors 2.8 GBmodel-00004-of-00108.safetensors 2.8 GBmodel-00061-of-00108.safetensors 2.8 GBmodel-00057-of-00108.safetensors 2.8 GBmodel-00088-of-00108.safetensors 2.8 GBmodel-00095-of-00108.safetensors 2.8 GBmodel-00034-of-00108.safetensors 2.8 GBmodel-00007-of-00108.safetensors 2.8 GBmodel-00091-of-00108.safetensors 2.8 GBmodel-00106-of-00108.safetensors 2.8 GBmodel-00024-of-00108.safetensors 2.8 GBmodel-00060-of-00108.safetensors 2.8 GBmodel-00097-of-00108.safetensors 2.8 GBmodel-00027-of-00108.safetensors 2.8 GBmodel-00056-of-00108.safetensors 2.8 GBmodel-00102-of-00108.safetensors 2.8 GBmodel-00033-of-00108.safetensors 2.8 GBmodel-00054-of-00108.safetensors 2.8 GBmodel-00015-of-00108.safetensors 2.8 GBmodel-00051-of-00108.safetensors 2.8 GBmodel-00065-of-00108.safetensors 2.8 GBmodel-00085-of-00108.safetensors 2.8 GBmodel-00039-of-00108.safetensors 2.7 GBmodel-00038-of-00108.safetensors 2.7 GBmodel-00005-of-00108.safetensors 2.7 GBmodel-00083-of-00108.safetensors 2.7 GBmodel-00093-of-00108.safetensors 2.7 GBmodel-00035-of-00108.safetensors 2.7 GBmodel-00030-of-00108.safetensors 2.7 GBmodel-00086-of-00108.safetensors 2.7 GBmodel-00066-of-00108.safetensors 2.7 GBmodel-00012-of-00108.safetensors 2.7 GBmodel-00047-of-00108.safetensors 2.7 GBmodel-00008-of-00108.safetensors 2.7 GBmodel-00014-of-00108.safetensors 2.7 GBmodel-00063-of-00108.safetensors 2.7 GBmodel-00101-of-00108.safetensors 2.7 GBmodel-00089-of-00108.safetensors 2.7 GBmodel-00100-of-00108.safetensors 2.1 GBmodel-00103-of-00108.safetensors 2.0 GBmodel-00019-of-00108.safetensors 2.0 GBmodel-00046-of-00108.safetensors 1.9 GBmodel-00052-of-00108.safetensors 1.8 GBmodel-00090-of-00108.safetensors 1.7 GBmodel-00002-of-00108.safetensors 1.7 GBmodel-00022-of-00108.safetensors 1.6 GBmodel-00031-of-00108.safetensors 1.5 GBmodel-00050-of-00108.safetensors 1.5 GBmodel-00071-of-00108.safetensors 1.5 GBmodel-00108-of-00108.safetensors 1.4 GBmodel-00084-of-00108.safetensors 1.4 GBmodel-00081-of-00108.safetensors 1.4 GBmodel-00009-of-00108.safetensors 1.4 GBmodel-00017-of-00108.safetensors 1.4 GBmodel-00013-of-00108.safetensors 1.4 GBmodel-00075-of-00108.safetensors 1.4 GBmodel-00077-of-00108.safetensors 1.4 GBmodel-00068-of-00108.safetensors 1.4 GBmodel-00055-of-00108.safetensors 280 MB有文件?提交种子。
提交种子thinkingmachines/Inkling