Run JANG models in MLX Studio / vMLX

⚠️ Runtime: this bundle needs a vMLX (Python) build that is not released yet. It was built and validated on a vMLX development build that adds MiMo-V2.6 JANGHT loading (JANGT trellis + JANGH codebook experts on the MiMo-V2.6 omni model). Released vMLX builds do not load it. Use it with the first vMLX release that lists MiMo-V2.6 JANGHT support.

JANGQ-AI/MiMo-V2.6-Flash-MOPD-JANGHT2.5

MiMo-V2.6-Flash-MOPD for 128 GB Macs: the full omni model (text, image, video, audio in) in 93.9 GiB, with its native DFlash drafter in the box.

A JANGHT bundle of XiaomiMiMo/MiMo-V2.6-Flash-MOPD: a 310B-parameter MoE (48 layers, 256 routed experts top-8, layer 0 dense; 9 full-attention + 39 sliding-window layers with attention sinks) with vision, video and audio input.

• Routed experts: JANGHT, a per-unit mix of two formats, chosen by measurement:
  • JANGT: a trellis code at 2, 2.5 or 3 bits per weight (Hadamard-128 rotation with signs, 12-bit state, up to 8 exact fp16 outlier columns per projection).
  • JANGH: our Hadamard-32 codebook format at 2, 3 or 4 bits, rounded with GPTQ.
• Everything else in the text model: affine 8-bit (group 64) — attention, the dense layer-0 MLP, embeddings and lm_head. Routers, expert-score bias and attention sinks stay fp32.
• Vision tower, audio encoder, speech embeddings: bf16, unchanged from the source. Audio tokenizer: the encoder half, verbatim (the decoder only produces speech and is not used for input).
• DFlash drafter (dflash/): MiMo's native block-diffusion draft model (MIT), 2-D weights affine 8-bit.
• No MTP: the three MTP layers are omitted (num_nextn_predict_layers: 0); DFlash replaces them.
• Name: JANGHT + 2.5 bits per weight averaged over the whole model (91.84 GiB of model shards × 8 / 310.76B source parameters = 2.539), drafter and audio tokenizer excluded.

Quality vs the original model

254 held-out sequences, 153,581 teacher-forced positions, scored against the original checkpoint (official MXFP4 experts decoded exactly, FP8 attention dequantized per block, bf16 elsewhere) run layer by layer from disk. Top-128 renormalized KL. Most positions are prompts and conversations from public datasets (multi-turn tool use, image and audio questions, long documents); every assistant turn was written by MiMo-V2.6-Flash-MOPD itself (T=1.0, top-p 0.95, thinking on and off), so on those turns agreement measures how often the quant makes the same next-token choice as the original model on its own text.

How to read this:

• MiMo is sensitive to tiny changes on long, high-entropy text. Rounding only the non-expert weights to 8 bits, with the experts untouched, already drops top-1 agreement to 69.6% overall and to 58–67% on image-conversation and tool-conversation text. No quant of this model reaches 100% on those sets; the floor rows are the realistic ceiling. Affine 8-bit beat MXFP8 on every domain and is what ships.
• On reasoning, chat and security text the bundle stays close: 91–93% top-1 agreement, median KL 0.001–0.007.
• Next-token accuracy on the actual held-out text: 34.9% for the bundle vs 36.0% for the original model.

By domain, median KL · mean KL · top-1 agreement:

Tool decisions

Inside the 153k positions: 89 points where the original model's own transcript starts a tool call, 149 where it starts the answer (first token after </think>).

Reasoning ends on its own

At the 231 points where the original model closes its reasoning (</think>) in its own transcripts:

Every expert unit carries the unit-gain row scales (the fix for low-bit reasoning that never closes, see our GLM-5.3-Flash JANGH2 revision 2).

Free generation (served, thinking on, T=1.0, top-p 0.95, 4 hard prompts × 3 seeds, 12k-token cap): 11 / 12 runs closed their reasoning by themselves with no repetition, after 268 to 9,003 characters of reasoning. The twelfth (the 23-person birthday problem, seed 2) was still multiplying the 22 factors out by hand, digit by digit and without repeating itself, when it reached the 12k-token cap; the other two seeds of that prompt closed. Re-run alone with a 32k-token cap, that prompt and seed still had not closed after 32,000 reasoning tokens. Long hand-arithmetic derivations can overrun the reasoning budget; set a generous max_tokens.

Live behavior (served, vMLX, plain lane)

• The two tool-use misses are the same case: asked "how many kilometers is 26.2 miles?" with thinking on, the model called calculator with 26.2 * 1.609344 instead of convert_units.
• Agent episodes: 160 tool calls, 1 repeat (page 2 of a search fetched twice, then a correct summary). One episode ignored a tool error that said the exchange was closed and answered that the ticker was not real. The test-fix loop (run tests → read → edit → re-run until green), schema exploration and a 5-item to-do were completed in every run.
• Audio transcription of a spoken sentence: "The quick brown fox jumps over the lazy dog near the river bank." — exact.

Speed (M5 Max 128 GB)

Served, one request at a time, plain lane, median of 3 probes on never-seen prompts (one server run).

All numbers on this card are from the plain lane. vMLX support for the bundled DFlash drafter is in progress; until it ships, serve with --no-bundled-dflash2.

What's in the bundle

• Experts (47 MoE layers; a unit is the gate+up pair or the down projection of one layer):

  projection	JANGT 2	JANGT 2.5	JANGT 3	JANGH 2	JANGH 3	JANGH 4
  gate / up	20 layers	22	2	2	—	1
  down	8	24	—	12	1	2

  Layers 5 and 16 are all-JANGH; the per-layer map is in config.json → quantization and jang_config.json → expert_units.

  Per-layer formats (measured from the shards)

  Layer 0 is the dense MLP (affine 8-bit). Type = attention (full, or sliding window 128 with sinks). Outlier = exact fp16 columns per JANGT projection.

  layer	type	gate	up	down	outlier g/u/d	GiB
  1	SWA-128	JANGT2	JANGT2	JANGT2	8/8/8	1.557
  2	SWA-128	JANGT2	JANGT2	JANGT2	8/8/8	1.557
  3	SWA-128	JANGT2	JANGT2	JANGT2	8/8/8	1.557
  4	SWA-128	JANGT2	JANGT2	JANGT2	8/8/8	1.557
  5	full	JANGH2	JANGH2	JANGH2	–/–/–	1.504
  6	SWA-128	JANGH2	JANGH2	JANGT2	–/–/8	1.531
  7	SWA-128	JANGT2	JANGT2	JANGT2	8/8/8	1.557
  8	SWA-128	JANGT2	JANGT2	JANGT2.5	8/8/8	1.682
  9	SWA-128	JANGT2	JANGT2	JANGH2	5/5/–	1.523
  10	SWA-128	JANGT3	JANGT3	JANGH4	3/3/–	2.520
  11	full	JANGT2.5	JANGT2.5	JANGH3	2/2/–	2.018
  12	SWA-128	JANGT2	JANGT2	JANGH2	1/1/–	1.516
  13	SWA-128	JANGT2	JANGT2	JANGT2.5	0/0/8	1.668
  14	SWA-128	JANGT2	JANGT2	JANGT2.5	0/0/8	1.668
  15	SWA-128	JANGT2	JANGT2	JANGH2	0/0/–	1.516
  16	SWA-128	JANGH4	JANGH4	JANGH4	–/–/–	3.004
  17	full	JANGT2	JANGT2	JANGT2.5	0/0/8	1.668
  18	SWA-128	JANGT2	JANGT2	JANGH2	0/0/–	1.516
  19	SWA-128	JANGT2	JANGT2	JANGT2.5	0/0/8	1.668
  20	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	0/0/8	1.918
  21	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	0/0/8	1.918
  22	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  23	full	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  24	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  25	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  26	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  27	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  28	SWA-128	JANGT2.5	JANGT2.5	JANGT2	2/2/8	1.795
  29	full	JANGT2.5	JANGT2.5	JANGT2.5	2/2/8	1.920
  30	SWA-128	JANGT2	JANGT2	JANGH2	2/2/–	1.518
  31	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	2/2/8	1.920
  32	SWA-128	JANGT2	JANGT2	JANGH2	1/1/–	1.516
  33	SWA-128	JANGT2	JANGT2	JANGH2	2/2/–	1.518
  34	SWA-128	JANGT2	JANGT2	JANGH2	1/1/–	1.516
  35	full	JANGT2	JANGT2	JANGH2	2/2/–	1.518
  36	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	0/0/8	1.918
  37	SWA-128	JANGT2	JANGT2	JANGH2	1/1/–	1.516
  38	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	0/0/8	1.918
  39	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  40	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	2/2/8	1.920
  41	full	JANGT3	JANGT3	JANGT2	2/2/8	2.045
  42	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	2/2/8	1.920
  43	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	2/2/8	1.920
  44	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	2/2/8	1.920
  45	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  46	SWA-128	JANGT2.5	JANGT2.5	JANGT2.5	1/1/8	1.918
  47	full	JANGT2.5	JANGT2.5	JANGH2	1/1/–	1.766

• Model shards: 18 shards, 91.84 GiB. Routed experts 83.94 GiB; text non-experts affine 8-bit; vision tower, audio encoder and speech embeddings bf16.

• audio_tokenizer/ (0.61 GiB): encoder only, verbatim.

• dflash/ (1.45 GiB): DFlash drafter — 5 sliding-window (1024) layers reading the target's hidden states from layers 0 / 11 / 23 / 35 / 47, block size 8, sharing the target's embeddings and lm_head; 36 linears affine 8-bit g64, mask_embedding.safetensors (+ the original .pt), MIT.

• Thinking + agentic:

  • Thinking is ON by default; pass chat_template_kwargs: {"enable_thinking": false} to turn it off. There are no reasoning-effort levels.
  • Tool calls use MiMo's XML dialect (<tool_call><function=name><parameter=key>value</parameter></function></tool_call>), declared as tool_parser: xml_function; reasoning as think_xml.
  • Past assistant turns keep their reasoning_content in the template; send it back in multi-turn agent loops.

• Self-describing:

  • config.json carries the per-module quantization map: 120 JANGT expert projections ("mode": "jangt", "kbits"), 21 JANGH ("mode": "jangtq2", "bits", "rotation": "hadamard32"), affine 8-bit (group 64) for every other quantized linear. The jangt block holds the trellis parameters and the per-projection outlier columns; the jangtq block (version 2) holds the JANGH codebooks.
  • jang_config.json records the calibration, the allocation plan, the per-unit formats, the drafter and the capabilities.

• Every tensor file is alignment-safe (zero-copy memory mapping), including the drafter and audio tokenizer.

Serving contract

• Sampling: temperature=1.0, top_p=0.95 (model card defaults), no repetition penalty
• EOS: [151643, 151645, 151672] · context: 1M declared (memory, not the model, sets the practical limit on 128 GB)
• Thinking: enable_thinking chat-template kwarg, default on
• JANGH units keep their on-disk identifiers (jangtq block, "mode": "jangtq2", *.tq2_packed / *.tq2_scales). They are not loadable by, and must not be routed to, JANGTQ v1 loaders.

Build details

• Source: XiaomiMiMo/MiMo-V2.6-Flash-MOPD @ b12f557; experts decoded exactly from the official MXFP4 (QAT) weights
• Calibration: 1.29M tokens. Every assistant turn was generated by the source model itself (364k generated tokens, thinking on and off), over coding in 9 languages, multi-round agentic tool use, tool conversations, terminal / shell, cybersecurity (two public instruction and vulnerability-fix datasets), reasoning, chat, Chinese, prose, 165 images and 60 audio clips run through the bf16 towers. Held out: 254 sequences (154k tokens) never used for calibration
• Allocation: per unit (gate+up or down of each layer), the format and width with the lowest predicted cost: measured held-out error curves weighted by each layer's measured sensitivity (real quantization error, frozen routing)
• Rounding: JANGT trellis on per-expert input statistics with outlier columns and column scaling on down projections; JANGH GPTQ with act-order in the rotated basis; unit-gain row scales on every unit
• Drafter: MiMo's native DFlash drafter, weights rounded to affine 8-bit; architecture unchanged

Quantized and validated by Jinho Jang — eric@jangq.ai