⚡ All JANG models are meant to be run in vMLX
DeepSeek-V4-Flash-JANGTQ-K
JANGTQ-K — the canonical max-quality JANG quantization of deepseek-ai/DeepSeek-V4-Flash for Apple Silicon, in the ~80 GiB tier.
| Source | deepseek-ai/DeepSeek-V4-Flash |
| License | MIT, inherited from upstream |
| Format | JANG (MXTQ routed + affine non-routed, critical controls F32) |
| Profile | JANGTQ_K |
| Bundle size | ~80 GiB (85.9 GB) across 80 shards |
| Tensor keys | 2610 |
| Context length | 1,048,576 (1M) |
| Native cache schema | deepseek_v4_v7 |
What this is
The first DSV4-Flash JANGTQ bundle that survives the relaxed multi-turn reasoning_effort=max quality-chat gate. It pairs:
- 2-bit MXTQ routed experts on 38 of 43 layers,
- 4-bit MXTQ routed experts on the 5 layers that lost the most quality at 2-bit (L23, L25, L28, L34, L36), selected from a real-activation probe over all 43 layers,
- 8-bit affine for every non-routed module (attention, shared expert, Compressor, Indexer, embed, lm_head),
- float32 preserved for HC controls, attention sinks, and APE,
- MTP head dropped (DSV4-Flash ships one MTP module of ~6.5 B params; not used at inference).
Live Quality
Passed: 4-turn reasoning_effort=max chat gate. Correct arithmetic, memory recall (remembered HARBOR-17 and CERULEAN), final 3-bullet format honored, no loops, no empty visible answers. Paged DSV4 cache hit on turns 2–4 with 399 tokens saved per hit; block-L2 wrote composite DSV4 blocks. Speed: 14.6 – 17.6 tok/s on M3 Ultra Mac Studio.
Not yet passed: strict exact-copy gate. Short exact-marker rows can flip to neighboring BPE tokens. The failure reproduces with prefix-cache bypass and is not a UI, gateway, streaming, or disk-cache artifact. Raw-max (VMLINUX_DSV4_RAW_MAX=1) had a 2-turn smoke only and is not advertised as fully cleared.
Quantization Recipe
| Category | Bits | Codec | Notes |
|---|---|---|---|
| Routed experts (38 layers default) | 2 | MXTQ | hash layers L0/L1/L2 included at 2-bit |
| Routed experts (5 lifted layers) | 4 | MXTQ | L23, L25, L28, L34, L36 |
Attention wq_a/wq_b/wkv/wo_a/wo_b |
8 | affine, gsz=32 | |
| Shared expert | 8 | affine, gsz=32 | |
| Compressor + Indexer + Indexer.Compressor | 8 | affine, gsz=32 | |
embed_tokens + lm_head |
8 | affine, gsz=32 | |
Norms / router gate / hc_* fn matrices |
16 | passthrough | |
hc_*_base, hc_*_scale, attn_sink, ape |
32 | source-f32 | |
| MTP head | — | dropped | drop_mtp=true; num_nextn_predict_layers=0 |
JANG vs MLX comparison
Benchmarks pending. To follow Eric's README standard (no TBD), this section will be filled in before any social promotion. The intended axes are:
- MMLU (reasoning + no-reasoning)
- HumanEval pass@1
- Korean / Chinese subset accuracy
- Decode tok/s on M3 Ultra, M5 Max
- Bundle size on disk
The MLX comparison baseline will be the smallest MLX-affine quant of DSV4-Flash that loads on the same hardware; if MLX has no published DSV4 quant at this size class, that absence will be stated.
Architecture (for downstream tooling)
- 43 decoder layers, no MTP.
- MoE: 256 routed + 1 shared per layer, top-6,
routed_scaling_factor=1.5,topk_method=noaux_tc. - MLA:
q_lora_rank=1024,o_lora_rank=1024,head_dim=512,qk_rope_head_dim=64. - Sparse Indexer:
index_n_heads=64,index_head_dim=128,index_topk=512,compress_rope_theta=160000. - HC controls:
hc_mult=4,hc_sinkhorn_iters=20,hc_eps=1e-6. - Sliding window 128; SwiGLU clip 10.0;
rope_theta=10000. - Context length 1,048,576 with YaRN.
- Reasoning modes
chat/thinking;reasoning_effort∈ {max,high,null}. - Tool calling: DSML parser (
<｜DSML｜>block).
Runtime contract
Bundle metadata pins:
- DSV4 native model family
deepseek_v4, batch pathDSV4BatchGenerator. - Cache schema
deepseek_v4_v7; layers 0,1 =KVCache, 2..42 =DeepseekV4Cache. - Generic TurboQuant KV: off for DSV4 (composite cache).
- DSV4 pool quant: off by default; opt-in only with
DSV4_POOL_QUANT=1. - Paged cache block size: 256 (loader upgrades stale 64-token settings).
- Block disk L2: on.
- Chunked prefill: off; single-shot prefill is the safe path.
JANGTQ runtime sidecar
jangtq_runtime.safetensors carries the JANGTQ MXTQ codebook + signs tables required by the Swift loader. Six tensors total (codebook.{2048,4096}.{2,4} + signs.{2048,4096}.42).
Sampling defaults
temperature=0.6, top_p=0.95
repetition_penalty=1.0 (1.0 thinking, 1.05 chat)
max_new_tokens=4096
Files
config.json— DSV4 HF config.jang_config.json— JANG profile, recipe, bit plan, runtime requirements, chat encoder, sampling defaults, lineage.model-00001-of-00080.safetensors … model-00080-of-00080.safetensors— sharded weights.model.safetensors.index.json— tensor → shard map (2610 keys).jangtq_runtime.safetensors— JANGTQ MXTQ runtime sidecar.tokenizer.json,tokenizer_config.json— preserved upstream tokenizer.generation_config.json— HF defaults.encoding/encoding_dsv4.py— DSV4 chat encoder (Python).LICENSE— MIT, upstream.
Lineage
Converter variant V3 (now collapsing into K), plan routed_only_worst5_23_25_28_34_36 (sha256 3db0b31fe6f1b19d3e00cfdd15572ebf3af950ef25e1e8622e1f2791b1977619). Originally staged as DeepSeek-V4-Flash-JANGTQ-V3-WORST5-F32; renamed to JANGTQ-K on 2026-05-11 as the canonical DSV4 max-quality tier.