Runtime: Osaurus 0.25.21 or newer (
osaurus.json:required_osaurus_version).
This bundle uses the JANGH expert format (
jangtq2banks); it needs a runtime with the JANGH MoE kernels.
OsaurusAI/LFM2.5-24B-A2B-JANGH2
A JANG bundle of LiquidAI/LFM2.5-24B-A2B: Liquid AI's 24B mixture-of-experts reasoning model with 2B active parameters (40 layers: 30 short-convolution and 10 full-attention, 64 routed experts with 4 active, two dense layers), built for on-device agents. It always thinks before answering and calls tools in Liquid's Pythonic format. 44.7 GiB of bf16 weights in 7.3 GiB.
What is quantized how
| part | format |
|---|---|
| routed experts gate / up | JANGH 2-bit: Hadamard-32 rotation + odd-cubic codebook, fp16 per-row scale, act-order GPTQ per expert |
| routed experts down | JANGH 3-bit, same scheme |
| dense MLP (layers 0-1), attention q/k/v/out, short-conv in/out projections | affine 8-bit, GPTQ |
| embeddings (tied lm_head) | affine 8-bit |
| routers, expert bias, depthwise conv kernels, norms | source values |
Every quantized matrix with an input Hessian was rounded with GPTQ. The Hessians come from the model's own outputs: the bf16 model wrote its thinking and answers for math, finance, code in eleven languages, terminal and coding-agent sessions (with real tool results), multi-step tool calls, networking, cybersecurity, science, office tasks, long documents and multilingual chat at its native sampling settings, and those sequences were the calibration data. Per-expert Hessians use only the tokens routed to that expert. Affine quantization is MLX's own grid (bf16 scales and biases).
Fidelity vs the bf16 model
232,094 teacher-forced positions on held-out sequences the bf16 model generated itself (never seen in calibration), scored against the bf16 model: top-128 KL, top-1 agreement, and the change in log-likelihood of the real next token (dNLL).
This model routes chaotically: changing the bf16 embeddings by a single bf16 ULP already moves top-1 on 1.8% of positions. That run is the floor row below; read the quantized rows against it.
| size | KL median | KL mean | top-1 | dNLL | |
|---|---|---|---|---|---|
| bf16 model, embeddings perturbed by 1 ULP (the floor) | 44.7 GiB | 1.1e-04 | 0.0036 | 98.24% | -0.0000 |
| JANGH2 | 7.3 GiB | 5.5e-03 | 0.1136 | 89.93% | +0.0989 |
By prompt family (KL median, top-1):
| family | KL median | top-1 |
|---|---|---|
| multi-step tool agent | 0.0195 | 86.3% |
| tool use (single turn) | 0.0212 | 88.9% |
| code: cpp | 0.0015 | 91.1% |
| code: go | 0.0000 | 94.6% |
| code: javascript | 0.0014 | 91.0% |
| code: python | 0.0002 | 93.5% |
| code: rust | 0.0081 | 89.3% |
| code: shell | 0.0111 | 89.0% |
| code: swift | 0.0072 | 89.8% |
| code: typescript | 0.0029 | 90.8% |
| creative writing | 0.0289 | 85.8% |
| cybersecurity | 0.0525 | 83.4% |
| finance | 0.0068 | 90.1% |
| general / multilingual / structured output | 0.0127 | 87.9% |
| coding agent (native tools) | 0.0017 | 93.7% |
| coding agent, next turn | 0.0059 | 93.5% |
| long documents | 0.0150 | 87.6% |
| math | 0.0010 | 92.6% |
| networking | 0.0014 | 91.9% |
| office tasks | 0.0043 | 91.8% |
| PyTorch code | 0.0033 | 90.4% |
| science | 0.0034 | 90.9% |
| SWE-agent | 0.0313 | 84.7% |
| terminal agent (shell) | 0.0038 | 90.5% |
| turn after a tool result | 0.0279 | 87.7% |
Decisions: the quantized model picks the same token as bf16 at 98 / 99 </think> closes (floor run: 99 / 99) and at 22 / 24 tool-call starts (floor run: 24 / 24).
Model notes
- Text only. 131,072-token context.
- Reasoning: always on. The chat template opens
<think>for every answer; there is no off switch and no effort levels.preserve_thinkingkeeps earlier turns' thinking in the context. - Tools: tool definitions go into the system prompt; calls come back as
<|tool_call_start|>[name(arg='value')]<|tool_call_end|>(a Python-literal list, parsed as data). Tool results use thetoolrole. - Sampling (
generation_config.json, mirrored injang_config.json): temperature 0.1, top-k 50, repetition penalty 1.05, EOS<|im_end|>. - Router: sigmoid scores with the expert bias used only to pick the top-4; the expert weights are the unbiased scores. The bundle's
jang_config.jsondocuments this; a softmax router produces wrong output on this model. - No draft model exists for the 24B, so there is no speculative decoder in this bundle.
Source
LiquidAI/LFM2.5-24B-A2Brevision1769516ac588, Apache-2.0.- Quantization and bundle by Jinho Jang (eric@osaurus.ai), 2026-10-13.