Runtime: Osaurus 0.25.27 or newer (
osaurus.json:required_osaurus_version0.25.27,model_version1).
OsaurusAI/DeepSeek-V4.1-Flash-1017-JANGHT1.4
DeepSeek-V4.1-Flash for 128 GB Macs. 94.8 GiB of the model stays in memory (the model 92.2 GiB, its DSpark speculative drafter 2.6 GiB). The Engram n-gram tables (203 GB) stay on the SSD at the release's full precision and are read per token. Image input, tool calls and thinking with three reasoning efforts are supported.
A JANGHT bundle of deepseek-ai/DeepSeek-V4.1-Flash-1017. The source is a 40-layer MoE (hidden 5120) with 384 routed experts per layer (top-6 plus a shared expert), compressed sparse attention with a 128-token local window, two Engram n-gram layers, a vision tower, and a 3-stage DSpark drafter. About 552B parameters outside the Engram tables; the routed experts are 98 % of them.
- Routed experts: JANGT trellis codes, Hadamard-128 rotation, per-row gains refit on the activations, exact outlier columns. Widths were allocated per layer and projection from each layer's measured effect on the assistant-token KL: gate/up 1 bit in 33 layers, 1.5 in 6, 2 in 1; down 1 bit in 5 layers, 1.5 in 22, 2 in 12, 2.5 in 1.
- Attention, shared experts, compressor and Engram projections: MXFP8, a bit-exact repack of the release's FP8 weights (no requantization).
- Token embeddings and
lm_head: affine 8-bit (group 64). - Vision tower: affine 8-bit linears; patch embedding, norms and biases bf16.
- Engram n-gram tables: the release shards byte for byte (
engram/), never resident. - DSpark drafter (
dspark/, 3 stages): experts JANGT 1 bit (gate/up) and 1.5 bits (down), attention and shared experts MXFP8.
Bits/weight 1.433 = resident model bytes × 8 / source parameters, with the Engram tables and the drafter excluded from both (the name rounds it to 1.4).
Quality vs the source model
Teacher-forced on held-out conversations that were never used for calibration (118 conversations, 157,086 scored positions), the bundle exactly as it ships against the source model. KL is over the source's top-128 tokens plus one tail bucket. Noise floor = the source model scored against itself with different batching, the best any build can measure.
Assistant tokens (what the model writes)
| held-out domain | positions | mean KL ↓ | median KL ↓ | p99 KL ↓ | top-1 ↑ | top-5 ↑ | noise floor: KL / top-1 |
|---|---|---|---|---|---|---|---|
| agentic (tool calls, multi-turn) | 20,282 | 0.109 | 0.017 | 1.08 | 89.4 % | 99.8 % | 0.0021 / 98.4 % |
| coding | 31,915 | 0.161 | 0.0014 | 3.01 | 91.1 % | 98.9 % | 0.0015 / 99.1 % |
| office | 4,809 | 0.124 | 0.0006 | 1.94 | 92.4 % | 99.5 % | 0.0008 / 99.3 % |
| reasoning (high / max effort) | 33,873 | 0.139 | 0.0085 | 1.72 | 89.0 % | 99.4 % | 0.0013 / 98.9 % |
| terminal | 7,556 | 0.063 | 0.013 | 0.64 | 90.8 % | 99.9 % | 0.0009 / 98.9 % |
| vision (image questions) | 4,493 | 0.153 | 0.047 | 1.38 | 85.6 % | 99.5 % | 0.0026 / 98.1 % |
| all | 102,928 | 0.134 | 0.0069 | 1.87 | 89.9 % | 99.4 % | 0.0015 / 98.9 % |
Every scored position (system prompts, tool schemas, tool results and user text included): mean KL 0.262, median 0.016, top-1 86.4 %, top-5 98.1 % (noise floor 0.0148 / 97.6 %).
- The flips sit where the source itself is unsure. Where the source puts ≥ 0.99 on its top token (51 % of assistant tokens) the bundle agrees 99.3 % of the time; at 0.9–0.99, 96.6 %; at 0.5–0.9, 83.3 %; below 0.5, 54.9 %.
- Tool calls are nearly exact. DSML tool-call tokens: top-1 99.6 %, KL 0.004. Most of the agentic KL (71 %) sits on tool-schema text in the system prompt, text the model reads rather than writes.
- Only the routed experts cost anything. Scoring with only the experts quantized gives the same numbers, so the MXFP8, 8-bit embedding / head and 8-bit vision parts are free.
- Cybersecurity text was not in the calibration or held-out sets.
Reasoning
- Thinking closes normally. At held-out
</think>positions the bundle's probability of closing is on average 2.1× below the source's. Most of that comes from very short thinks (one repeated 17-token agentic pattern); by reasoning length the ratio is 1.01× at 200–800 tokens, 0.96× at 800–2,000, 1.13× at 2,000–4,000 and 2.9× past 4,000 (4 closes). There is no close where the source is above 0.5 and the bundle below 0.05. - A row-gain correction (scaling each expert row back up toward the source's output norm) was tested and not
applied: it raised the assistant KL from 0.135 to 0.217–0.446 and suppressed
</think>more, not less. - A hard coding prompt (a Rust IPv4 parser with unit tests, effort high and max, 3 seeds, 8,192-token budget) closed in
3 of 6 runs at 3.1k–6.0k tokens. Replaying those runs through the source model showed the same behavior: the source
closes at the same point and continues the same long runs of zeros where two runs looped. Give long coding questions
a generous
max_tokens.
Raw results: evaluation/kl_vs_source_heldout.json, evaluation/reasoning_close.json,
evaluation/dspark_acceptance.json.
DSpark speculative decoding
The bundle ships the release's DSpark drafter (3 stages, 5-token blocks), quantized as above. Verification is exact: greedy output equals plain decoding (up to bf16 ties), and sampled output follows the same distribution as plain sampling (the draft only proposes; the target's own sample decides every token).
Tokens committed per verify round, teacher-forced on the held-out set:
| coding | reasoning | office | terminal | agentic | all | |
|---|---|---|---|---|---|---|
| this bundle | 3.60 | 3.48 | 3.94 | 2.78 | 2.70 | 3.16 |
| source drafter on the source model | 3.80 | 3.65 | 4.20 | 2.93 | 2.83 | 3.33 |
Speed and memory (Osaurus engine, M5 Max 128 GB)
Measured on the engine's production generation path, 300-token replies, thinking off, medians of the prompts in each group, two interleaved runs (ranges show both).
| decode, tok/s | essays | code (LRU cache, Rust parser, TS debounce) | repetitive code | repetitive text |
|---|---|---|---|---|
| plain decoding | 28.6–29.9 | 28.6–29.7 | 28.6–29.8 | 28.7–29.8 |
| DSpark, greedy | 28.4–30.3 | 52.4–55.8 | 66.4–66.5 | 68.7–69.3 |
| DSpark, temperature 1.0 (the release default) | 28.1 | 51.6 | 66.1 | 68.8 |
| cold prefill (drafter resident) | 607 tok/s at 5.6k tokens, 751 tok/s at 23.2k |
| time to first token, short prompt | ~0.5 s |
| load | ~17–24 s |
| peak memory footprint while decoding | 102.9 GB with DSpark (100.0 GB plain) |
DSpark speeds up code about 1.8× and repetitive output about 2.3×, and matches plain decoding on free-form prose: the engine times both and only speculates while it is faster.
What's in the bundle
- Vision: the vision tower and its aligner, in-bundle (
config.json→vision_config). - DSpark drafter:
dspark/dspark.safetensors; on by default in the engine. - Engram:
engram/(two release shards, 203 GB on disk) plusengram_aux.safetensorsat the root; rows are read from the SSD per token, so keep the bundle on a fast internal or Thunderbolt drive. - Thinking + agentic:
- Thinking is ON by default;
enable_thinking: falseturns it off. - Reasoning effort
low/high/max(defaulthigh) via thereasoning_effortchat-template kwarg. - Tool calls in the release's DSML format, declared as
tool_parser: deepseek_v41; reasoning parserdeepseek_r1. encoding/is the release's reference prompt encoder.
- Thinking is ON by default;
- Self-describing:
config.jsoncarries thejangtexpert-format block and a per-modulequantizationmap;jang_config.jsoncarries the stamps (capabilities, chat, sampling, reasoning efforts, drafter, Engram) and the per-layer expert widths.jang_config.formatreadsjangtq2for loader compatibility; every routed expert ismode: jangt. - Every shard is alignment-safe (zero-copy memory mapping);
model.safetensors.index.jsoncovers the 43 model shards.
Serving contract
- Sampling: the release defaults,
temperature=1.0, top_p=1.0 - EOS:
[1, 128803, 128804, 128828]· context: 1,048,576 - Reasoning:
reasoning_effort=low/high/max(defaulthigh)
Build details
- Source:
deepseek-ai/DeepSeek-V4.1-Flash-1017(revision1bae0e1) - Calibration: 1.53M tokens of reasoning traces at high and max effort, multi-language coding, terminal and multi-turn agentic tool use, office work and image questions; held-out sets disjoint
- Experts: JANGT trellis (1 / 1.5 / 2 / 2.5 bits per weight per layer and projection), Hadamard-128 rotation, per-row gains refit in the activation metric, exact outlier columns; widths from measured per-layer sensitivity
- Everything else: the release's FP8 as MXFP8 (bit-exact), affine 8-bit embeddings,
lm_headand vision linears - Engram tables and DSpark from the release; the drafter's acceptance on this bundle is 95 % of the source's
Quantized and validated by Jinho Jang — eric@osaurus.ai