⚡ All JANG models are meant to be run in vMLX

DeepSeek-V4.1-Flash-1017-JANG_2K

Mixed-affine JANG quantization of deepseek-ai/DeepSeek-V4.1-Flash-1017 for Apple Silicon, with the release's Engram n-gram tables, its 3-stage dSpark speculative draft and its vision tower all included. Built and measured on a 256 GB M5 Ultra.

Quantization (JANG_2K)

JANG_2K spends one extra bit on the routed down_proj (its output enters the residual stream) and keeps every non-routed path at 8-bit. The first eight layers also get 3-bit gate/up.

Routed-expert average: 2.47 bits. Routed experts were requantized from the FP4 source with an importance-matrix clip search (activation-x²-weighted, 11 clip ratios per group, plain affine always among the candidates) using statistics from 694,517 tokens of technical and security-domain text. MTP weights of the main model are replaced by the bundled dSpark draft.

Quality

Measured on this exact bundle. Teacher-forced perplexity on 32 held-out rows (6,766 tokens), excluded from all calibration statistics:

The three builds are within 1.3% of each other, which is within the noise of 6.8k tokens. Top-1 agreement and KL against the original FP4/FP8 checkpoint are not reported: the source model does not run on Apple Silicon, so there is no reference on the same hardware yet.

Runtime fidelity checks (real model):

• vMLX speed paths are exact: prompt-suffix pruning gives last-token logits cos 0.99996 / 0.99994 at 8k / 32k with the same top-5 and 24 of 24 greedy tokens; the fused decode kernels give 256 of 256 identical greedy tokens; an identical answer with pruning on and off at 924k tokens.
• dSpark output equals greedy decoding up to near-ties (it runs one verify forward per step).
• Long-context retrieval, greedy, thinking off, question at ~54 % depth:
  • one distinctive fact in ~460k / ~845k tokens of natural text: correct both times (top-1 probability 0.99);
  • 46,200 / 92,600 numbered lines (Doc line N: value V, random 3-digit values; 461k / 925k tokens): correct (0.90 / 0.99);
  • a deliberately hard variant (IDs like r1m-15617 plus a second number on every line, space-joined) is correct up to 300k tokens and misses at 461k / 924k. The miss is the same with cold, RAM and SSD caches, with 512- or 2048-token prefill chunks, with bf16 cache pools and with the release's own FP8/FP4 activation rounding; and putting the release's original FP4 routed experts back in layers 0-19 does not bring the target line into the sparse-attention candidate set (it ranks lower, not higher). We read it as a limit of the model's sparse attention on near-identical lines, not of this quantization or runtime.

Sampling defaults

generation_config.json sets DeepSeek's recommended sampling (temperature 1.0, top_p 1.0). vMLX (Python and Swift / Osaurus) applies it when a request sends no sampling parameters; dSpark then runs its exact sampled path. Send temperature: 0 for greedy output.

Speed and memory (M5 Ultra 256 GB, thinking off, unique-text cold prompts)

Prefill above 2.4k tokens is fast because layers 21–39 only compute the prompt rows the last token can still reach through their 128-token windows (exact). Decode is fast because the per-layer work runs in a few fused Metal dispatches. The compressed KV pools are stored 4-bit, as in the reference's FP4 cache.

Runtime requirement

This bundle needs a DeepSeek-V4.1-aware runtime: Engram tables read from engram/, the compressed sparse attention with shared indexer, hyper-connections, the dSpark draft and the vision tower. It does not load in stock mlx-lm.

• vMLX (Python): the jang_tools.dsv41 runtime (jjang-ai/jangq) with vMLX DeepSeek-V4.1 serving support: vmlx serve <bundle>. dSpark is on by default for greedy requests (VMLX_DSV41_DSPARK=0 turns it off).
• vMLX Swift / Osaurus: DeepSeek-V4.1 support in osaurus-ai/vmlx-swift (PR #657) and Osaurus (PR #3206).

Keep the bundle on a fast internal SSD: every prompt token reads 48 random Engram rows.

Chat contract

The official release ships encoding/encoding.py (no Jinja template); vMLX renders with it. V4.1 reasoning effort is numeric (Reasoning Effort: N); chat_template_kwargs.enable_thinking=false gives the non-reasoning mode. Tool calls use V4.1 DSML and are returned as OpenAI tool_calls. Images go in standard OpenAI image_url content parts.

Download

hf download JANGQ-AI/DeepSeek-V4.1-Flash-1017-JANG_2K \
  --local-dir ~/models/DeepSeek-V4.1-Flash-1017-JANG_2K

Korean summary

공식 deepseek-ai/DeepSeek-V4.1-Flash-1017 릴리스를 Apple Silicon용 JANG_2K 혼합 affine(라우티드 전문가 평균 2.47비트, down 3비트, 비라우티드 8비트, imatrix 보정)으로 양자화한 번들입니다. Engram n-gram 테이블(SSD에서 행 단위로 읽음), 3단계 dSpark 추측 디코딩 드래프트, 비전 타워가 모두 포함되어 있습니다. 256 GB M5 Ultra 기준 vMLX Python에서 dSpark 코드 디코드 약 90 tok/s, 100k 토큰 콜드 프리필 약 1,584 tok/s, 1M 토큰 KV 캐시 약 905 MiB, 메모리 약 207 GB를 측정했습니다. 원본 체크포인트 대비 top-1/KL은 동일 하드웨어 기준 참조가 없어 아직 보고하지 않습니다.

Contact

eric@jangq.ai