Thinking Machines Inkling (975B total / 41B active MoE) converted to the colibri expert-streaming container, for CPU-first inference on a single machine.
Quantized from the original BF16 checkpoint (not the NVFP4 release), so the routed experts take a single quantization step: bf16 → int4.
...mlp.experts.gate_up_proj / down_proj tensor is packed U8
(low nibble = even column, offset +8) with a <name>.qs f32 sidecar of row scales —
the standard colibri container read by glm.c/inkling.c.w13 tensors were de-interleaved (gate/up rows alternate in the source
checkpoint) and renamed to the HF layout, matching transformers' conversion mapping.Total size ≈ 469 GB. Runtime RSS ≈ 90 GB (bf16 residents) + expert cache.
Requires the inkling branch of nbeerbower/colibri
(Stage-A engine; upstreaming planned):
git clone -b inkling https://github.com/nbeerbower/colibri
cd colibri/c && make inkling
SNAP=/path/to/this/repo ./inkling -p "The capital of France is" -n 64
The engine validates bit-exact against HF transformers on a tiny oracle
(tools/make_tiny_inkling.py), and this snapshot was produced by
tools/convert_inkling_int4.py (threaded, resumable, shard-at-a-time).
tok.h handles it natively (357/357
encode-exact vs HF tokenizers in testing).out-mtp.safetensors 11 GBout-00044.safetensors 5.8 GBout-00026.safetensors 5.5 GBout-00092.safetensors 5.5 GBout-00049.safetensors 5.5 GBout-00011.safetensors 5.5 GBout-00105.safetensors 5.5 GBout-00001.safetensors 5.5 GBout-00072.safetensors 5.5 GBout-00059.safetensors 5.5 GBout-00036.safetensors 5.5 GBout-00021.safetensors 5.4 GBout-00042.safetensors 5.4 GBout-00076.safetensors 5.4 GBout-00045.safetensors 5.4 GBout-00069.safetensors 5.4 GBout-00104.safetensors 5.4 GBout-00094.safetensors 5.4 GBout-00037.safetensors 5.4 GBout-00032.safetensors 5.4 GBout-00080.safetensors 5.3 GBout-00003.safetensors 5.3 GBout-00028.safetensors 5.3 GBout-00087.safetensors 5.3 GBout-00023.safetensors 5.3 GBout-00082.safetensors 5.3 GBout-00053.safetensors 5.2 GBout-00025.safetensors 5.2 GBout-00040.safetensors 5.2 GBout-00018.safetensors 5.2 GBout-00079.safetensors 5.2 GBout-00016.safetensors 5.2 GBout-00070.safetensors 5.2 GBout-00020.safetensors 5.2 GBout-00067.safetensors 5.2 GBout-00058.safetensors 5.2 GBout-00006.safetensors 5.1 GBout-00088.safetensors 5.1 GBout-00029.safetensors 5.1 GBout-00048.safetensors 5.1 GBout-00078.safetensors 5.1 GBout-00073.safetensors 5.1 GBout-00057.safetensors 5.1 GBout-00098.safetensors 5.1 GBout-00096.safetensors 5.1 GBout-00099.safetensors 5.1 GBout-00010.safetensors 5.1 GBout-00061.safetensors 5.1 GBout-00043.safetensors 5.1 GBout-00004.safetensors 5.1 GBout-00041.safetensors 5.1 GBout-00007.safetensors 5.0 GBout-00095.safetensors 5.0 GBout-00034.safetensors 5.0 GBout-00089.safetensors 5.0 GBout-00106.safetensors 5.0 GBout-00097.safetensors 5.0 GBout-00102.safetensors 5.0 GBout-00033.safetensors 5.0 GBout-00091.safetensors 5.0 GBout-00024.safetensors 5.0 GBout-00060.safetensors 5.0 GBout-00027.safetensors 5.0 GBout-00064.safetensors 5.0 GBout-00056.safetensors 5.0 GBout-00054.safetensors 5.0 GBout-00085.safetensors 4.9 GBout-00015.safetensors 4.9 GBout-00051.safetensors 4.9 GBout-00065.safetensors 4.9 GBout-00107.safetensors 4.9 GBout-00039.safetensors 4.9 GBout-00038.safetensors 4.9 GBout-00005.safetensors 4.9 GBout-00083.safetensors 4.9 GBout-00093.safetensors 4.9 GBout-00035.safetensors 4.9 GBout-00030.safetensors 4.9 GBout-00086.safetensors 4.8 GBout-00066.safetensors 4.8 GBout-00047.safetensors 4.8 GBout-00012.safetensors 4.8 GBout-00062.safetensors 4.8 GBout-00008.safetensors 4.8 GBout-00014.safetensors 4.8 GBout-00063.safetensors 4.8 GBout-00074.safetensors 4.8 GBout-00101.safetensors 4.8 GBout-00046.safetensors 4.0 GBout-00090.safetensors 3.6 GBout-00052.safetensors 3.5 GBout-00002.safetensors 3.3 GBout-00022.safetensors 3.0 GBout-00050.safetensors 2.9 GBout-00100.safetensors 2.9 GBout-00031.safetensors 2.8 GBout-00071.safetensors 2.8 GBout-00108.safetensors 2.6 GBout-00084.safetensors 2.6 GBout-00009.safetensors 2.6 GBout-00081.safetensors 2.5 GBout-00017.safetensors 2.5 GBout-00103.safetensors 2.5 GBout-00013.safetensors 2.5 GBout-00075.safetensors 2.4 GBout-00077.safetensors 2.4 GBout-00068.safetensors 2.4 GBout-00019.safetensors 2.4 GBout-00055.safetensors 717 MB有文件?提交种子。
提交种子thinkingmachines/Inkling