The performance / maximum-fidelity tier of the Inkling · Alis MLX Dynamic family — a two-box pipeline build shipped as ready-to-serve stage artifacts. Siblings: the 3.7 bpw quality build (single 512 GB Mac) and the 2.7 bpw capacity build. Full family: Inkling 975B · Alis MLX Dynamic collection.
Apple Silicon (MLX) Q6 quantization of thinkingmachines/Inkling — a 975B-class multimodal Mixture-of-Experts model (66 hybrid decoder layers, 256 routed experts (top-6) + 2 shared per MoE layer, hidden 6144, sliding-window attention + short-convolution hybrid, vision + audio front-ends, 201K vocab).
This build answers the question the other tiers can't: what does Inkling sound like when you stop trading anything away? ~6.60 bits/weight, 728 GiB across two 512 GB Macs — routed experts at a near-transparent 6-bit, and every other tensor in the model byte-exact BF16. In the Alis pipeline this artifact is the maximum-fidelity reference tier: the closest distributable approximation of the BF16 model.
| Component | Precision | Tensors | Why |
|---|---|---|---|
Routed experts w13/w2 (layers 3–65) |
6-bit affine, g128 | 126 | the parameter bulk, at near-transparent width |
| Everything else — attention, routers/gates, shared experts, dense MLP, embeddings, LM head, norms, vision + audio towers, layers 0–2 | BF16 (byte-exact) | 1,266 | nothing on any control or modality path is quantized |
Dropped: the source's MTP head (
model.mtp.*, 5.26B params) — a speculative-decoding accessory not used by the MLX runtime. All three family tiers drop it identically, which is why they report 947B logical params vs the source's 952B.
Quantized with the alis-clip quantizer (per-group clipping search from the alis-dwq toolchain) rather than plain rounding, with per-shard checksum verification receipts at write time.
| Base model | thinkingmachines/Inkling (975B-class; 952.4B source / 947.1B logical params) |
| Bits/weight | 6.60 effective (781.2 GB total) |
| On-disk size | 781.2 GB (727.6 GiB) — stage0 363 GiB + stage1 365 GiB |
| Format | MLX safetensors (inkling_mm_model), two pipeline-stage artifacts |
| Modalities | text + image + audio inputs |
| Serving | 2 × 512 GB Apple Silicon, MLX pipeline parallelism |
This repo ships the model the way a two-box pipeline actually consumes it:
stage0/ # pipeline rank 1 — layers 0–33 + embeddings + vision/audio towers (103 shards)
stage1/ # pipeline rank 0 — layers 33–66 + final norm + LM head (109 shards)
The repo-root config.json is the full-model reference config (partition-free); the loadable artifacts are the stage folders. Each stage folder is a self-contained MLX artifact (own config.json + weight index) whose config declares its pipeline partition. The loader prunes to stage-owned tensors and strict-checks the partial index — each host downloads and verifies only its ~365 GiB half:
# host A (rank 1) # host B (rank 0)
from inkling_mlx.model import load_inkling_model
model, cfg = load_inkling_model( model, cfg = load_inkling_model(
".../stage0", ".../stage1",
pipeline_group=group) pipeline_group=group)
with group = mx.distributed.init() over the two hosts (the Inkling MLX runtime's pipeline serving path; upstream mlx-lm support for inkling_mm_model is not yet merged — treat this repo as weights + provenance until the port lands upstream).
The Inkling · Alis DWQ tiers were trained against exact BF16 teacher activations dumped by a distributed two-Mac pipeline; this Q6 build is the same pipeline's maximum-fidelity distributable tier, and serves as the high-precision reference in the project's held-out evaluation gates. Unlike its siblings it carries no DWQ pass — at 6-bit with clip-search and a fully BF16 skeleton, the conversion is already near the noise floor the DWQ contract measures against.
| Build | bpw | Size | For |
|---|---|---|---|
| quality / golden spot | 3.71 | 409 GiB | single 512 GB Mac, best quality in one box |
| capacity / size-optimal | 2.72 | 299 GiB | 320 GB+ Macs, maximum headroom |
| this — performance / two-box Q6 | 6.60 | 728 GiB | 2 × 512 GB pipeline, maximum fidelity |
Both stage conversions are complete with sealed per-shard receipts (conversion_state.json: complete, alis-clip, per-shard checksum verification at write). The project's remaining gates — two-box candidate held-out evaluation (which uses this build as the Q6 reference) and final quality validation — are in progress; this card will be updated with their measurements as they land.
ml-explore.inkling_mm_model port: Alis (avlp12).Alis (avlp12) (2026). Inkling-975B-Alis-MLX-Dynamic-6.6bpw — two-box Q6 MLX quantization of Inkling. https://huggingface.co/avlp12/Inkling-975B-Alis-MLX-Dynamic-6.6bpw
stage0/model-00107-of-00108.stage0.safetensors 19 GBstage0/model-00001-of-00108.stage0.safetensors 10 GBstage0/model-00080-of-00108.stage0.safetensors 8.1 GBstage1/model-00003-of-00108.stage1.safetensors 8.0 GBstage1/model-00072-of-00108.stage1.safetensors 8.0 GBstage1/model-00105-of-00108.stage1.safetensors 8.0 GBstage1/model-00026-of-00108.stage1.safetensors 7.9 GBstage1/model-00082-of-00108.stage1.safetensors 7.9 GBstage1/model-00069-of-00108.stage1.safetensors 7.9 GBstage0/model-00025-of-00108.stage0.safetensors 7.9 GBstage1/model-00076-of-00108.stage1.safetensors 7.9 GBstage0/model-00020-of-00108.stage0.safetensors 7.9 GBstage0/model-00058-of-00108.stage0.safetensors 7.9 GBstage0/model-00067-of-00108.stage0.safetensors 7.9 GBstage0/model-00029-of-00108.stage0.safetensors 7.9 GBstage1/model-00036-of-00108.stage1.safetensors 7.8 GBstage1/model-00104-of-00108.stage1.safetensors 7.8 GBstage0/model-00053-of-00108.stage0.safetensors 7.8 GBstage1/model-00023-of-00108.stage1.safetensors 7.8 GBstage0/model-00094-of-00108.stage0.safetensors 7.8 GBstage1/model-00021-of-00108.stage1.safetensors 7.8 GBstage0/model-00070-of-00108.stage0.safetensors 7.8 GBstage1/model-00011-of-00108.stage1.safetensors 7.8 GBstage1/model-00087-of-00108.stage1.safetensors 7.8 GBstage0/model-00095-of-00108.stage0.safetensors 7.8 GBstage0/model-00041-of-00108.stage0.safetensors 7.7 GBstage1/model-00073-of-00108.stage1.safetensors 7.7 GBstage1/model-00097-of-00108.stage1.safetensors 7.7 GBstage1/model-00057-of-00108.stage1.safetensors 7.7 GBstage0/model-00045-of-00108.stage0.safetensors 7.7 GBstage1/model-00098-of-00108.stage1.safetensors 7.7 GBstage1/model-00018-of-00108.stage1.safetensors 7.7 GBstage1/model-00034-of-00108.stage1.safetensors 7.7 GBstage1/model-00043-of-00108.stage1.safetensors 7.7 GBstage1/model-00010-of-00108.stage1.safetensors 7.7 GBstage0/model-00099-of-00108.stage0.safetensors 7.7 GBstage0/model-00015-of-00108.stage0.safetensors 7.7 GBstage0/model-00048-of-00108.stage0.safetensors 7.7 GBstage1/model-00037-of-00108.stage1.safetensors 7.7 GBstage0/model-00040-of-00108.stage0.safetensors 7.7 GBstage0/model-00065-of-00108.stage0.safetensors 7.7 GBstage1/model-00054-of-00108.stage1.safetensors 7.7 GBstage0/model-00088-of-00108.stage0.safetensors 7.6 GBstage1/model-00039-of-00108.stage1.safetensors 7.6 GBstage1/model-00038-of-00108.stage1.safetensors 7.6 GBstage0/model-00032-of-00108.stage0.safetensors 7.6 GBstage0/model-00079-of-00108.stage0.safetensors 7.6 GBstage0/model-00007-of-00108.stage0.safetensors 7.6 GBstage1/model-00006-of-00108.stage1.safetensors 7.6 GBstage1/model-00106-of-00108.stage1.safetensors 7.6 GBstage1/model-00085-of-00108.stage1.safetensors 7.6 GBstage0/model-00078-of-00108.stage0.safetensors 7.6 GBstage1/model-00083-of-00108.stage1.safetensors 7.6 GBstage0/model-00060-of-00108.stage0.safetensors 7.6 GBstage1/model-00093-of-00108.stage1.safetensors 7.6 GBstage0/model-00024-of-00108.stage0.safetensors 7.6 GBstage1/model-00030-of-00108.stage1.safetensors 7.6 GBstage0/model-00056-of-00108.stage0.safetensors 7.6 GBstage0/model-00061-of-00108.stage0.safetensors 7.6 GBstage1/model-00016-of-00108.stage1.safetensors 7.6 GBstage0/model-00066-of-00108.stage0.safetensors 7.6 GBstage0/model-00004-of-00108.stage0.safetensors 7.5 GBstage1/model-00012-of-00108.stage1.safetensors 7.5 GBstage1/model-00086-of-00108.stage1.safetensors 7.5 GBstage1/model-00062-of-00108.stage1.safetensors 7.5 GBstage0/model-00027-of-00108.stage0.safetensors 7.5 GBstage1/model-00005-of-00108.stage1.safetensors 7.5 GBstage0/model-00014-of-00108.stage0.safetensors 7.5 GBstage0/model-00035-of-00108.stage0.safetensors 7.5 GBstage0/model-00051-of-00108.stage0.safetensors 7.5 GBstage0/model-00063-of-00108.stage0.safetensors 7.5 GBstage1/model-00008-of-00108.stage1.safetensors 7.5 GBstage1/model-00074-of-00108.stage1.safetensors 7.5 GBstage1/model-00091-of-00108.stage1.safetensors 7.5 GBstage0/model-00101-of-00108.stage0.safetensors 7.5 GBstage0/model-00046-of-00108.stage0.safetensors 4.8 GBstage0/model-00002-of-00108.stage0.safetensors 4.5 GBstage1/model-00044-of-00108.stage1.safetensors 4.4 GBstage1/model-00090-of-00108.stage1.safetensors 4.4 GBstage0/model-00059-of-00108.stage0.safetensors 4.4 GBstage1/model-00052-of-00108.stage1.safetensors 4.3 GBstage1/model-00092-of-00108.stage1.safetensors 4.3 GBstage1/model-00049-of-00108.stage1.safetensors 4.2 GBstage0/model-00042-of-00108.stage0.safetensors 4.2 GBstage1/model-00028-of-00108.stage1.safetensors 4.2 GBstage0/model-00100-of-00108.stage0.safetensors 4.1 GBstage1/model-00001-of-00108.stage1.safetensors 4.1 GBstage1/model-00022-of-00108.stage1.safetensors 4.1 GBstage0/model-00049-of-00108.stage0.safetensors 4.0 GBstage0/model-00096-of-00108.stage0.safetensors 4.0 GBstage1/model-00050-of-00108.stage1.safetensors 4.0 GBstage0/model-00092-of-00108.stage0.safetensors 4.0 GBstage0/model-00084-of-00108.stage0.safetensors 4.0 GBstage1/model-00102-of-00108.stage1.safetensors 3.9 GBstage0/model-00031-of-00108.stage0.safetensors 3.9 GBstage1/model-00042-of-00108.stage1.safetensors 3.9 GBstage0/model-00033-of-00108.stage0.safetensors 3.9 GBstage1/model-00009-of-00108.stage1.safetensors 3.9 GBstage0/model-00064-of-00108.stage0.safetensors 3.9 GBstage0/model-00089-of-00108.stage0.safetensors 3.9 GBstage0/model-00071-of-00108.stage0.safetensors 3.9 GBstage0/model-00108-of-00108.stage0.safetensors 3.9 GBstage0/model-00028-of-00108.stage0.safetensors 3.9 GBstage0/model-00103-of-00108.stage0.safetensors 3.9 GBstage1/model-00096-of-00108.stage1.safetensors 3.8 GBstage0/model-00017-of-00108.stage0.safetensors 3.8 GBstage1/model-00075-of-00108.stage1.safetensors 3.8 GBstage1/model-00059-of-00108.stage1.safetensors 3.8 GBstage1/model-00013-of-00108.stage1.safetensors 3.8 GBstage0/model-00102-of-00108.stage0.safetensors 3.8 GBstage1/model-00081-of-00108.stage1.safetensors 3.8 GBstage1/model-00047-of-00108.stage1.safetensors 3.8 GBstage1/model-00033-of-00108.stage1.safetensors 3.8 GBstage0/model-00068-of-00108.stage0.safetensors 3.8 GBstage0/model-00047-of-00108.stage0.safetensors 3.8 GBstage0/model-00077-of-00108.stage0.safetensors 3.8 GBstage1/model-00019-of-00108.stage1.safetensors 3.8 GBstage1/model-00064-of-00108.stage1.safetensors 3.8 GBstage0/model-00044-of-00108.stage0.safetensors 2.7 GBstage1/model-00089-of-00108.stage1.safetensors 2.5 GBstage0/model-00055-of-00108.stage0.safetensors 604 MBstage0/model-00052-of-00108.stage0.safetensors 579 MBstage1/model-00046-of-00108.stage1.safetensors 535 MBstage0/model-00090-of-00108.stage0.safetensors 529 MBstage0/model-00037-of-00108.stage0.safetensors 502 MBstage1/model-00032-of-00108.stage1.safetensors 463 MBstage0/model-00011-of-00108.stage0.safetensors 456 MBstage1/model-00045-of-00108.stage1.safetensors 431 MBstage0/model-00021-of-00108.stage0.safetensors 403 MBstage1/model-00094-of-00108.stage1.safetensors 343 MBstage0/model-00016-of-00108.stage0.safetensors 327 MBstage0/model-00036-of-00108.stage0.safetensors 315 MBstage0/model-00104-of-00108.stage0.safetensors 305 MBstage1/model-00040-of-00108.stage1.safetensors 302 MBstage1/model-00079-of-00108.stage1.safetensors 289 MBstage0/model-00076-of-00108.stage0.safetensors 277 MBstage0/model-00026-of-00108.stage0.safetensors 277 MBstage0/model-00022-of-00108.stage0.safetensors 277 MBstage1/model-00078-of-00108.stage1.safetensors 264 MBstage0/model-00006-of-00108.stage0.safetensors 255 MBstage0/model-00050-of-00108.stage0.safetensors 252 MBstage0/model-00087-of-00108.stage0.safetensors 252 MBstage0/model-00105-of-00108.stage0.safetensors 245 MBstage0/model-00069-of-00108.stage0.safetensors 239 MBstage1/model-00061-of-00108.stage1.safetensors 239 MBstage1/model-00071-of-00108.stage1.safetensors 239 MBstage1/model-00004-of-00108.stage1.safetensors 227 MBstage1/model-00031-of-00108.stage1.safetensors 227 MBstage1/model-00088-of-00108.stage1.safetensors 227 MBstage0/model-00018-of-00108.stage0.safetensors 227 MBstage0/model-00072-of-00108.stage0.safetensors 214 MBstage1/model-00002-of-00108.stage1.safetensors 179 MBstage0/model-00023-of-00108.stage0.safetensors 179 MBstage1/model-00048-of-00108.stage1.safetensors 179 MBstage0/model-00091-of-00108.stage0.safetensors 167 MBstage1/model-00007-of-00108.stage1.safetensors 151 MBstage1/model-00053-of-00108.stage1.safetensors 151 MBstage0/model-00106-of-00108.stage0.safetensors 151 MBstage1/model-00099-of-00108.stage1.safetensors 142 MBstage0/model-00010-of-00108.stage0.safetensors 126 MBstage1/model-00070-of-00108.stage1.safetensors 126 MBstage1/model-00027-of-00108.stage1.safetensors 126 MBstage1/model-00024-of-00108.stage1.safetensors 126 MBstage0/model-00081-of-00108.stage0.safetensors 126 MBstage0/model-00057-of-00108.stage0.safetensors 120 MBstage0/model-00062-of-00108.stage0.safetensors 118 MBstage1/model-00108-of-00108.stage1.safetensors 113 MBstage0/model-00043-of-00108.stage0.safetensors 113 MBstage1/model-00056-of-00108.stage1.safetensors 113 MBstage0/model-00073-of-00108.stage0.safetensors 113 MBstage1/model-00055-of-00108.stage1.safetensors 113 MBstage1/model-00060-of-00108.stage1.safetensors 104 MBstage0/model-00098-of-00108.stage0.safetensors 104 MBstage1/model-00051-of-00108.stage1.safetensors 101 MBstage1/model-00100-of-00108.stage1.safetensors 101 MBstage1/model-00017-of-00108.stage1.safetensors 101 MBstage1/model-00107-of-00108.stage1.safetensors 101 MBstage0/model-00085-of-00108.stage0.safetensors 76 MBstage0/model-00082-of-00108.stage0.safetensors 76 MBstage0/model-00005-of-00108.stage0.safetensors 75 MBstage1/model-00025-of-00108.stage1.safetensors 63 MBstage0/model-00013-of-00108.stage0.safetensors 63 MBstage0/model-00034-of-00108.stage0.safetensors 50 MBstage1/model-00041-of-00108.stage1.safetensors 38 MBstage0/model-00083-of-00108.stage0.safetensors 25 MBstage0/model-00003-of-00108.stage0.safetensors 25 MBstage1/model-00067-of-00108.stage1.safetensors 25 MBstage1/model-00035-of-00108.stage1.safetensors 13 MBstage0/model-00054-of-00108.stage0.safetensors 13 MBstage1/model-00080-of-00108.stage1.safetensors 3 MBstage1/model-00020-of-00108.stage1.safetensors 3 MBstage1/model-00084-of-00108.stage1.safetensors 3 MBstage0/model-00086-of-00108.stage0.safetensors 3 MBstage0/model-00009-of-00108.stage0.safetensors 3 MBstage0/model-00039-of-00108.stage0.safetensors 224.4 kBstage1/model-00058-of-00108.stage1.safetensors 31.2 kBstage1/model-00029-of-00108.stage1.safetensors 17.7 kBstage0/model-00012-of-00108.stage0.safetensors 17.7 kBstage0/model-00075-of-00108.stage0.safetensors 17.7 kBstage0/model-00030-of-00108.stage0.safetensors 17.7 kBstage1/model-00077-of-00108.stage1.safetensors 13.6 kBstage0/model-00038-of-00108.stage0.safetensors 2.3 kBstage1/model-00066-of-00108.stage1.safetensors 2.3 kB有文件?提交种子。
提交种子thinkingmachines/Inkling