Runtime: Osaurus 0.25.18 or newer (osaurus.json: required_osaurus_version 0.25.18, model_version 1).

OsaurusAI/Allosaurus-v0.1-125B-A6B-JANGH2

Qwen3.8-Flash-Next for 48 GB Macs. About 35 GiB of the model lives in memory; the 54 GiB n-gram embedding table is read from the SSD on demand. Against our previous smallest release (JANG_1L, 41 GiB in memory) it is 6 GiB smaller, with 3x lower median KL to the bf16 model and +9 points top-1, and it keeps tool calls and reasoning closes on track where the previous release slipped.

A JANGH bundle of Qwen/Qwen3.8-Flash-Next: a 125B-parameter MoE with 512 routed experts (top-10 + a shared expert), gated-delta-net and sparse attention layers, one MTP layer, and image + video input, plus a 51B-entry n-gram embedding table. About 6.7B parameters are active per token (not counting embedding lookups).

• Routed experts: JANGH 2-bit. Codebook-quantized with per-row scales and a blockwise Hadamard rotation, calibrated with GPTQ on each expert's own input covariance.
• Everything else in memory: affine 4-bit (linear-attention projections, token embeddings) or 8-bit (sparse attention, shared experts, lm_head); routers in bf16; all calibrated (importance matrix + AWQ).
• n-gram embedding table: affine 8-bit, kept on the SSD and read per token (16 rows × 180 bytes).
• Vision tower: 6-bit, calibrated on 985 images and video clips (earlier Flash-Next releases quantized it uncalibrated).
• MTP layer: 4-bit, calibrated from its own capture (earlier releases quantized it uncalibrated).

JANGH

JANGH replaces our earlier JANGTQ (TurboQuant) expert format: a fixed blockwise Hadamard-32 transform (no random signs) and a two-constant codebook per bit width, stored bit-for-bit in MLX's own packing layout. On disk the identifiers keep their original names: config.json → jangtq block (version: 2), per-module "mode": "jangtq2", tensors *.tq2_packed / *.tq2_scales. These are the JANGH format and are not loadable by JANGTQ v1 loaders.

Quality vs the bf16 model

514,498 teacher-forced positions on four held-out sets never used for calibration, scored against the full bf16 model run layer by layer from disk. Top-128 renormalized KL, scored on the bundle exactly as stored.

Read this first: some next tokens are nearly tied, so any change to the weights flips a few of them. We measured that on the unquantized model (relative noise of 0.001 on a single tensor). That run is the ceiling below; the distance to the ceiling, not to 100%, is what a quantization costs.

General set by domain, median KL · top-1 agreement:

Long documents are the one domain where the previous release stays closer to bf16; on the live long-context recall tests below this bundle still scores higher.

Decisions

Live behavior (served, greedy)

Recall, multi-key and variable tracking are perfect to 64k; counting markers degrades with length (the previous release too). Both bundles run out of an 8k token budget on the same seven long hand-arithmetic office items at low effort.

Reasoning efforts, same prompts (84 per effort; reference = our 6-bit release of the same model)

Low, medium and thinking-off match the 6-bit reference. At xhigh this 2-bit model reasons longer than the reference and more often reaches the 24k-token budget on long coding and verification tasks; it does not loop (no repetition in any unfinished trace), and on long reasoning that the reference closes, the probability it closes at the same point is 0.99983 (bf16: 0.99994; previous release lowest case 0.55). Give xhigh a generous max_tokens, or use medium.

MTP (speculative decoding)

The MTP proposal head ships as a calibrated 4-bit refit of this bundle's lm_head (in mtp_draft/, with its measured head description at the bundle root).

Speed and memory (Osaurus engine, M5 Max, pre-release build, median of 3)

Expert weights are memory-mapped from disk, so the process footprint stays at 7–9 GB while the OS pages the experts in. Decode after long prompts is the known gap in the pre-release engine (it recovers to 43.9 tok/s when the experts are held resident) and is being worked on; quality is unaffected.

What's in the bundle

• Vision + video: Qwen3-VL tower, image and video processor configs; video uses timestamped temporal groups.
• MTP: 1 layer, calibrated, plus the calibrated proposal head.
• Thinking + agentic:
  • Thinking is ON by default (the template opens <think>); preserve_thinking keeps reasoning in history.
  • Reasoning efforts low / medium / xhigh (default xhigh), via the reasoning_effort chat-template kwarg.
  • Thinking off (enable_thinking: false) renders <think>\n\n</think>\n\n.
• Tool calls: XML function calls (<tool_call>\n<function=name>\n<parameter=arg>…</parameter>\n</function>\n</tool_call>), declared as tool_parser: qwen; reasoning parser qwen3.
• Self-describing:
  • config.json carries the JANGH format block, a per-module quantization map, and jang_config with the stamps (chat, reasoning efforts, capabilities, MTP, context) and per-layer expert bits.
  • Raw evaluation results are in evaluation/.
• Every shard is alignment-safe (zero-copy memory mapping).

Serving contract

• Sampling: temperature=1.0, top_p=0.95, top_k=20 (vendor defaults), no repetition penalty
• EOS: [248046, 248044] · context: 262,144
• Reasoning: reasoning_effort = low / medium / xhigh (default xhigh)

Build details

• Source: Qwen/Qwen3.8-Flash-Next (bf16)
• Calibration: 607k tokens (code, security, agentic, general, science, systems, reasoning traces, tool conversations) with held-out sets kept disjoint; plus a dedicated MTP capture and a vision/video capture (985 images and clips: documents, screenshots, charts, code and terminal screenshots, invoices, real and screen-recorded video)
• Experts: JANGH 2-bit (odd-cubic codebook, fp16 per-row scale, Hadamard-32 rotation), GPTQ on all 48 MoE layers with each expert's centered input covariance; plain GPTQ scales (a unit-gain variant was A/B'd and lost on every held-out set)
• Resident weights: linear attention 4-bit, embeddings 4-bit, sparse attention 8-bit, shared experts 8-bit, lm_head 8-bit, routers bf16 — chosen by measured KL cost per GiB saved; importance-matrix refit + AWQ throughout
• n-gram embedding table: affine 8-bit (group 32), read from SSD

Quantized and validated by Jinho Jang — eric@osaurus.ai