An MLX 4-bit build of Thinking Machines' Inkling (975B-total / 41B-active MoE),
quantized from the BF16 checkpoint for Apple Silicon. Omni: keeps the text decoder
plus the vision (HMLP) and audio (dMel) towers. Self-contained. This bundles the loader
package (inkling_mlx/), so it runs with just mlx + mlx-lm + transformers.
The higher-fidelity sibling of mlx-community/Inkling-mlx-2bit:
~548 GB (vs ~315 GB at 2-bit), trading size for sharper text and multimodal quality.
experts_only)# pip install mlx mlx-lm transformers
# download this repo (it already includes the inkling_mlx/ loader)
from inkling_mlx.load import load
from inkling_mlx.generate import greedy_generate
from transformers import AutoTokenizer
model, config = load("./Inkling-mlx-4bit")
tok = AutoTokenizer.from_pretrained("./Inkling-mlx-4bit", trust_remote_code=True)
ids = tok("The capital of France is")["input_ids"]
print(tok.decode(greedy_generate(model, config, ids, max_new_tokens=64)))
At ~548 GB this doesn't fit resident on one Mac but an MoE only fires 6 of 256 experts per token. So keep the always-needed weights in RAM (attention, shared experts, embeddings, norms, router, vision / audio towers) and page the routed experts from SSD on demand, letting the OS page cache hold the hot ones. This runs the omni build on a single Mac Studio.
| Prompt | Generated continuation | Notes |
|---|---|---|
The capital of France is |
Paris. The capital of Italy is Rome. The capital of Spain is Madrid. The capital of Russia is Moscow. |
~0.33 tok/s, SSD expert-offload |
Tool: github.com/huckiyang/mlx-moe-offload
(MIT) — a drop-in over mlx-lm's SwitchGLU.
pip install mlx mlx-lm scipy pillow
git clone https://github.com/huckiyang/mlx-moe-offload && cd mlx-moe-offload && pip install -e .
# 1) one-time: repack the stacked experts into a per-expert SSD store
python -m mlx_moe_offload.repack --build /path/Inkling-mlx-4bit --out /path/Inkling-mlx-4bit-offload
OFF=/path/Inkling-mlx-4bit-offload
# 2) generate — text, image, or audio (omni)
python examples/inkling_omni.py --offload-dir $OFF --prompt "The capital of France is"
python examples/inkling_omni.py --offload-dir $OFF --image cat.png --prompt "What is in this image?"
python examples/inkling_omni.py --offload-dir $OFF --audio q.wav --prompt "Transcribe in English:"
Decode speed is gated by the routing cache hit-rate and SSD bandwidth, please check store.stats().
thinkingmachines/Inkling.inkling_mlx/ loader is vendored from
PipeNetwork/inkling-mlx (Apache-2.0,
Copyright 2026 PipeNetwork) and David.LICENSE + THIRD_PARTY_NOTICES.md are included in this repo.
model-00007-of-00066.safetensors 8.6 GBmodel-00008-of-00066.safetensors 8.6 GBmodel-00010-of-00066.safetensors 8.6 GBmodel-00014-of-00066.safetensors 8.6 GBmodel-00015-of-00066.safetensors 8.6 GBmodel-00019-of-00066.safetensors 8.6 GBmodel-00026-of-00066.safetensors 8.6 GBmodel-00028-of-00066.safetensors 8.6 GBmodel-00031-of-00066.safetensors 8.6 GBmodel-00032-of-00066.safetensors 8.6 GBmodel-00040-of-00066.safetensors 8.6 GBmodel-00041-of-00066.safetensors 8.6 GBmodel-00043-of-00066.safetensors 8.6 GBmodel-00044-of-00066.safetensors 8.6 GBmodel-00048-of-00066.safetensors 8.6 GBmodel-00050-of-00066.safetensors 8.6 GBmodel-00051-of-00066.safetensors 8.6 GBmodel-00053-of-00066.safetensors 8.6 GBmodel-00054-of-00066.safetensors 8.6 GBmodel-00039-of-00066.safetensors 8.6 GBmodel-00021-of-00066.safetensors 8.6 GBmodel-00035-of-00066.safetensors 8.6 GBmodel-00023-of-00066.safetensors 8.6 GBmodel-00063-of-00066.safetensors 8.6 GBmodel-00058-of-00066.safetensors 8.6 GBmodel-00006-of-00066.safetensors 8.6 GBmodel-00047-of-00066.safetensors 8.6 GBmodel-00004-of-00066.safetensors 8.6 GBmodel-00038-of-00066.safetensors 8.6 GBmodel-00011-of-00066.safetensors 8.6 GBmodel-00064-of-00066.safetensors 8.6 GBmodel-00046-of-00066.safetensors 8.6 GBmodel-00056-of-00066.safetensors 8.6 GBmodel-00027-of-00066.safetensors 8.6 GBmodel-00005-of-00066.safetensors 8.6 GBmodel-00052-of-00066.safetensors 8.6 GBmodel-00025-of-00066.safetensors 8.6 GBmodel-00033-of-00066.safetensors 8.6 GBmodel-00037-of-00066.safetensors 8.6 GBmodel-00018-of-00066.safetensors 8.6 GBmodel-00017-of-00066.safetensors 8.6 GBmodel-00020-of-00066.safetensors 8.6 GBmodel-00030-of-00066.safetensors 8.6 GBmodel-00060-of-00066.safetensors 8.6 GBmodel-00057-of-00066.safetensors 8.6 GBmodel-00024-of-00066.safetensors 8.6 GBmodel-00034-of-00066.safetensors 8.6 GBmodel-00059-of-00066.safetensors 8.6 GBmodel-00065-of-00066.safetensors 8.6 GBmodel-00013-of-00066.safetensors 8.6 GBmodel-00012-of-00066.safetensors 8.6 GBmodel-00061-of-00066.safetensors 8.6 GBmodel-00016-of-00066.safetensors 8.6 GBmodel-00029-of-00066.safetensors 8.6 GBmodel-00049-of-00066.safetensors 8.6 GBmodel-00062-of-00066.safetensors 8.6 GBmodel-00022-of-00066.safetensors 8.6 GBmodel-00055-of-00066.safetensors 8.6 GBmodel-00045-of-00066.safetensors 8.6 GBmodel-00009-of-00066.safetensors 8.6 GBmodel-00042-of-00066.safetensors 8.6 GBmodel-00036-of-00066.safetensors 8.6 GBmodel-00003-of-00066.safetensors 8.3 GBmodel-00043-of-00098.safetensors 6.4 GBmodel-00002-of-00098.safetensors 6.1 GBmodel-00033-of-00098.safetensors 6.1 GBmodel-00065-of-00098.safetensors 6.1 GBmodel-00023-of-00098.safetensors 6.1 GBmodel-00047-of-00098.safetensors 6.1 GBmodel-00066-of-00066.safetensors 6.0 GBmodel-00045-of-00098.safetensors 6.0 GBmodel-00010-of-00098.safetensors 6.0 GBmodel-00072-of-00098.safetensors 5.9 GBmodel-00042-of-00098.safetensors 5.9 GBmodel-00093-of-00098.safetensors 5.9 GBmodel-00028-of-00098.safetensors 5.9 GBmodel-00048-of-00098.safetensors 5.9 GBmodel-00029-of-00098.safetensors 5.9 GBmodel-00037-of-00098.safetensors 5.8 GBmodel-00081-of-00098.safetensors 5.8 GBmodel-00053-of-00098.safetensors 5.8 GBmodel-00050-of-00098.safetensors 5.8 GBmodel-00020-of-00098.safetensors 5.8 GBmodel-00063-of-00098.safetensors 5.8 GBmodel-00014-of-00098.safetensors 5.8 GBmodel-00083-of-00098.safetensors 5.8 GBmodel-00085-of-00098.safetensors 5.8 GBmodel-00024-of-00098.safetensors 5.7 GBmodel-00018-of-00098.safetensors 5.7 GBmodel-00073-of-00098.safetensors 5.7 GBmodel-00019-of-00098.safetensors 5.7 GBmodel-00080-of-00098.safetensors 5.7 GBmodel-00009-of-00098.safetensors 5.7 GBmodel-00046-of-00098.safetensors 5.7 GBmodel-00016-of-00098.safetensors 5.7 GBmodel-00095-of-00098.safetensors 5.7 GBmodel-00086-of-00098.safetensors 5.7 GBmodel-00038-of-00098.safetensors 5.7 GBmodel-00032-of-00098.safetensors 5.7 GBmodel-00004-of-00098.safetensors 5.7 GBmodel-00089-of-00098.safetensors 5.7 GBmodel-00039-of-00098.safetensors 5.7 GBmodel-00064-of-00098.safetensors 5.7 GBmodel-00017-of-00098.safetensors 5.7 GBmodel-00090-of-00098.safetensors 5.6 GBmodel-00070-of-00098.safetensors 5.6 GBmodel-00069-of-00098.safetensors 5.6 GBmodel-00041-of-00098.safetensors 5.6 GBmodel-00087-of-00098.safetensors 5.6 GBmodel-00062-of-00098.safetensors 5.6 GBmodel-00097-of-00098.safetensors 5.6 GBmodel-00005-of-00098.safetensors 5.6 GBmodel-00056-of-00098.safetensors 5.6 GBmodel-00040-of-00098.safetensors 5.6 GBmodel-00051-of-00098.safetensors 5.6 GBmodel-00044-of-00098.safetensors 5.6 GBmodel-00088-of-00098.safetensors 5.6 GBmodel-00022-of-00098.safetensors 5.6 GBmodel-00049-of-00098.safetensors 5.6 GBmodel-00052-of-00098.safetensors 5.6 GBmodel-00026-of-00098.safetensors 5.6 GBmodel-00066-of-00098.safetensors 5.6 GBmodel-00031-of-00098.safetensors 5.6 GBmodel-00078-of-00098.safetensors 5.6 GBmodel-00055-of-00098.safetensors 5.6 GBmodel-00068-of-00098.safetensors 5.6 GBmodel-00094-of-00098.safetensors 5.6 GBmodel-00082-of-00098.safetensors 5.6 GBmodel-00058-of-00098.safetensors 5.6 GBmodel-00079-of-00098.safetensors 5.5 GBmodel-00003-of-00098.safetensors 5.5 GBmodel-00059-of-00098.safetensors 5.5 GBmodel-00074-of-00098.safetensors 5.5 GBmodel-00076-of-00098.safetensors 5.5 GBmodel-00007-of-00098.safetensors 5.5 GBmodel-00025-of-00098.safetensors 5.5 GBmodel-00012-of-00098.safetensors 5.5 GBmodel-00034-of-00098.safetensors 5.5 GBmodel-00030-of-00098.safetensors 5.5 GBmodel-00075-of-00098.safetensors 5.5 GBmodel-00061-of-00098.safetensors 5.5 GBmodel-00035-of-00098.safetensors 5.5 GBmodel-00067-of-00098.safetensors 5.5 GBmodel-00057-of-00098.safetensors 5.5 GBmodel-00084-of-00098.safetensors 5.5 GBmodel-00071-of-00098.safetensors 5.5 GBmodel-00006-of-00098.safetensors 5.5 GBmodel-00021-of-00098.safetensors 5.4 GBmodel-00096-of-00098.safetensors 5.4 GBmodel-00008-of-00098.safetensors 5.4 GBmodel-00077-of-00098.safetensors 5.4 GBmodel-00011-of-00098.safetensors 5.4 GBmodel-00015-of-00098.safetensors 5.4 GBmodel-00092-of-00098.safetensors 5.4 GBmodel-00027-of-00098.safetensors 5.4 GBmodel-00091-of-00098.safetensors 5.4 GBmodel-00060-of-00098.safetensors 5.4 GBmodel-00036-of-00098.safetensors 5.4 GBmodel-00054-of-00098.safetensors 5.4 GBmodel-00013-of-00098.safetensors 5.4 GBmodel-00001-of-00098.safetensors 5.4 GBmodel-00002-of-00066.safetensors 5.2 GBmodel-00001-of-00066.safetensors 5.0 GBmodel-00098-of-00098.safetensors 415 MB有文件?提交种子。
提交种子thinkingmachines/Inkling