Run JANG models in MLX Studio / vMLX
Runtime: validated on a vMLX (Python) development build. DFlash2 speculative decoding requires vMLX with DFlash2 support.
JANGQ-AI/Qwen3.8-27B-JANG_4D
Qwen3.8-27B at near-reference quality in 21 GiB, with image and video input and a bundled DFlash2 drafter.
| part | format |
|---|---|
| MLP gate / up / down | affine 4 / 5 / 6 / 8-bit, picked per projection by measured error per byte |
| full attention (q / k / v / o) and gated-delta-net projections | affine 8-bit |
lm_head, token embeddings, vision tower |
affine 8-bit |
| norms, convolutions, gates | bf16 |
All affine modules: importance-matrix weighted fit, AWQ folded into the producing norm, GPTQ on the MLX grid below 8 bits.
Quality vs the bf16 model
| set | median KL | top-1 |
|---|---|---|
| mixed held-out text | 0.0007 | 95.2 % |
| agentic and tool conversations | 0.0014 | 92.1 % |
| reasoning | 0.0001 | 96.9 % |
| own bf16 reasoning traces, effort low | 0.0001 | 93.9 % |
| own bf16 reasoning traces, effort medium | 0.00004 | 97.7 % |
| own bf16 reasoning traces, effort xhigh | 0.0005 | 95.1 % |
Think-block close decisions agree with bf16 on 40 / 40 held-out traces and 65 / 65 text rows; tool-call vs answer decisions flip on 9 / 263 agentic positions.
Served checks (vMLX, parsers auto-selected from the bundle): thinking on and off, all three efforts, a tool call, an invoice image and a chart video all pass, with and without the DFlash2 drafter.
Quality by topic
Median KL and top-1 agreement vs the bf16 model per topic of the held-out text set (both bundles, same rows):
| topic | tokens scored | JANG_4D KL | JANG_4D top-1 | JANGH2 KL | JANGH2 top-1 |
|---|---|---|---|---|---|
| coding (many languages) | 39,155 | 0.0002 | 98.4 % | 0.0092 | 89.5 % |
| math | 9,067 | 0.0001 | 98.7 % | 0.0068 | 90.8 % |
| office / finance | 13,159 | 0.0001 | 97.7 % | 0.0087 | 86.9 % |
| reasoning | 14,004 | 0.0001 | 94.5 % | 0.0081 | 81.0 % |
| science | 4,156 | 0.0010 | 98.0 % | 0.0460 | 87.1 % |
| long context + attention recall | 79,784 | 0.0007 | 96.6 % | 0.0446 | 80.4 % |
| cybersecurity | 15,683 | 0.0008 | 94.9 % | 0.0368 | 80.5 % |
| networking | 6,458 | 0.0011 | 93.7 % | 0.0244 | 81.9 % |
| terminal / shell | 42,987 | 0.0013 | 92.5 % | 0.0738 | 73.5 % |
| agentic + tool calls | 34,245 | 0.0026 | 89.8 % | 0.0975 | 71.2 % |
| creative writing | 10,029 | 0.0028 | 96.5 % | 0.1989 | 74.2 % |
| multi-turn agentic set | 199,746 | 0.0014 | 92.1 % | 0.0615 | 75.3 % |
Own bf16 reasoning traces, by topic (thinking + answer tokens):
| topic | JANG_4D KL | JANG_4D top-1 | JANGH2 KL | JANGH2 top-1 |
|---|---|---|---|---|
| coding (many languages) | 0.00007 | 98.4 % | 0.00508 | 89.4 % |
| math | 0.00001 | 98.6 % | 0.00061 | 91.6 % |
| office / finance | 0.00000 | 98.5 % | 0.00007 | 91.2 % |
| science | 0.00010 | 98.1 % | 0.00779 | 87.8 % |
| networking | 0.00000 | 98.7 % | 0.00009 | 92.8 % |
| cybersecurity | 0.00135 | 97.3 % | 0.08516 | 81.1 % |
| agentic + tool calls | 0.00093 | 89.7 % | 0.03968 | 72.6 % |
Reasoning, tools, sampling
- Thinking is on by default (the template opens
<think>);enable_thinking: falseprefills a closed block. reasoning_effort: low, medium, xhigh (default); the template rejects anything else. Earlier thinking is kept in history by default (preserve_thinking).- Reasoning parser
qwen3; tool calls use the XML function format (parserqwen3_coder). - Sampling presets from the model card: thinking T 1.0 / top_p 0.95 / top_k 20; instruct T 0.7 / top_p 0.8 / top_k 20 / presence 1.5.
DFlash2 drafter
dflash2/ holds the z-lab/Qwen3.8-27B-DFlash2 block-diffusion
drafter (bf16, block size 8). The native MTP head is not included. In vMLX, pass
--speculative-model <bundle>/dflash2 --no-continuous-batching. The drafter is text-only; image and video requests
decode normally.
Calibration
One capture feeds the importance matrix, AWQ and GPTQ, merged by token count: 1.21M text tokens (cybersecurity, coding in many languages, agentic and tool conversations, terminal, long reasoning, attention recall, math; all three efforts), 0.43M image and video tokens (191 media items through the vision tower), and 0.31M tokens of the model's own full-precision reasoning traces at low / medium / xhigh. Held-out sets are disjoint from calibration.
What's in the bundle
model-*.safetensors + one index (64-byte aligned), config.json (per-module quantization), jang_config.json (format,
capabilities, reasoning and tool contract, sampling, evaluation), generation_config.json, tokenizer, chat template,
image and video processor configs, dflash2/.
Quantized and validated by Jinho Jang — eric@jangq.ai