Experimental low-bit conversion. Expect greater quality loss than the 4-bit and 5-bit versions.
Converted from the official Thinking Machines checkpoint, revision 6e7c8248f75265bdd4fbab5273304243b5d8c2cb.
The source is the authors' trained NVFP4/MXFP8 release, not an original BF16 checkpoint. Conversion to affine MLX adds reconstruction error; 5-bit conversion does not recover higher-precision training weights. This is an independent conversion, not an official Thinking Machines release.
1, .995, .985, .97, .94, .9, .85, .8, selected by reconstruction MSE. This is not activation-calibrated GPTQ or AWQ.The architecture has about 26.4B logical parameters, per the upstream model card. A Hub counter that counts packed integer storage may show a different number. The original auxiliary mtp.safetensors is not included or qualified by this conversion; no speculative-decoding speedup is claimed.
This model needs the pinned Inkling MLX-LM port. Stock released MLX-LM did not support this architecture when this artifact was prepared.
python3 -m venv inkling-env
source inkling-env/bin/activate
pip install 'mlx-lm @ git+https://github.com/ml-explore/mlx-lm.git@0c4c99192230f1943dd468d297f0b8a64ae7b7c2'
MLX_ENABLE_TF32=0 mlx_lm generate --model araysuter/Inkling-2-Mini-MLX-mixed-2bit --prompt 'Explain a binary search in two sentences.' -m 256
Validated with MLX 0.32.4, Python 3.14, and an Apple M5 Max with 64 GiB unified memory. The port's eight model tests pass with MLX_ENABLE_TF32=0. The default M5 reduced-precision float32 path fails three of those tests; MLX documents the precision switch. The recommended flag retains full float32 execution for those operations. This does not require converting all weights to float32.
For images, the pinned port includes the Inkling image processor and embed_inputs; the included validate_inkling.py shows the tested path with image embeddings. The full upstream context setting is retained in config; 128K and 1M contexts have not been qualified here.
All artifacts were reloaded from disk before these probes. Same tokenizer and text were used for the source and conversion; the native trained source ran with the same MLX port and precision setting.
| Check | Observed converted output |
|---|---|
| arithmetic | 391 |
| coding | python def larger(a, b): return max(a, b) |
| vision | There is a red square on the left and a blue circle on the right. |
| Synthetic text probe | Native source | This conversion |
|---|---|---|
| Mean negative log likelihood | 2.26608 | 2.15405 |
| Perplexity | 9.64151 | 8.61969 |
Delta NLL: -0.11203. This is one short, synthetic text passage plus three smoke prompts, not a standard benchmark or evidence of general benchmark accuracy. Full prompt text, responses, timing and checks are in VALIDATION.json. Reasoning tokens count toward the recorded generation lengths and speeds.
convert_inkling.py converts one profile at a time with bounded tensor tiles; it does not materialize a full BF16 checkpoint. Put the pinned original checkpoint in source/, then run MLX_ENABLE_TF32=0 python convert_inkling.py mixed-2bit. Use INKLING_WORKDIR to select the working directory. The included scripts use the runtime pinned above. QUANTIZATION_MANIFEST.json records every quantized matrix and its selected precision.
Original model: Thinking Machines Lab, Inc. Original license: Apache 2.0; see LICENSE, the upstream model card, and its linked Model Acceptable Use Policy. MLX runtime: the MLX contributors. Published quantized weights retain the upstream model license.
model-00001-of-00003.safetensors 5.4 GBmodel-00002-of-00003.safetensors 5.4 GBmodel-00003-of-00003.safetensors 2.5 GB