Runtime: Osaurus 0.25.21 or newer (osaurus.json: required_osaurus_version).

OsaurusAI/LFM2.5-24B-A2B-JANG_4M

A JANG bundle of LiquidAI/LFM2.5-24B-A2B: Liquid AI's 24B mixture-of-experts reasoning model with 2B active parameters (40 layers: 30 short-convolution and 10 full-attention, 64 routed experts with 4 active, two dense layers), built for on-device agents. It always thinks before answering and calls tools in Liquid's Pythonic format. 44.7 GiB of bf16 weights in 14.8 GiB.

What is quantized how

Every quantized matrix with an input Hessian was rounded with GPTQ. The Hessians come from the model's own outputs: the bf16 model wrote its thinking and answers for math, finance, code in eleven languages, terminal and coding-agent sessions (with real tool results), multi-step tool calls, networking, cybersecurity, science, office tasks, long documents and multilingual chat at its native sampling settings, and those sequences were the calibration data. Per-expert Hessians use only the tokens routed to that expert. Affine quantization is MLX's own grid (bf16 scales and biases).

Fidelity vs the bf16 model

232,094 teacher-forced positions on held-out sequences the bf16 model generated itself (never seen in calibration), scored against the bf16 model: top-128 KL, top-1 agreement, and the change in log-likelihood of the real next token (dNLL).

This model routes chaotically: changing the bf16 embeddings by a single bf16 ULP already moves top-1 on 1.8% of positions. That run is the floor row below; read the quantized rows against it.

By prompt family (KL median, top-1):

Decisions: the quantized model picks the same token as bf16 at 99 / 99 </think> closes (floor run: 99 / 99) and at 23 / 24 tool-call starts (floor run: 24 / 24).

Model notes

• Text only. 131,072-token context.
• Reasoning: always on. The chat template opens <think> for every answer; there is no off switch and no effort levels. preserve_thinking keeps earlier turns' thinking in the context.
• Tools: tool definitions go into the system prompt; calls come back as <|tool_call_start|>[name(arg='value')]<|tool_call_end|> (a Python-literal list, parsed as data). Tool results use the tool role.
• Sampling (generation_config.json, mirrored in jang_config.json): temperature 0.1, top-k 50, repetition penalty 1.05, EOS <|im_end|>.
• Router: sigmoid scores with the expert bias used only to pick the top-4; the expert weights are the unbiased scores. The bundle's jang_config.json documents this; a softmax router produces wrong output on this model.
• No draft model exists for the 24B, so there is no speculative decoder in this bundle.

Source

• LiquidAI/LFM2.5-24B-A2B revision 1769516ac588, Apache-2.0.
• Quantization and bundle by Jinho Jang (eric@osaurus.ai), 2026-10-13.