Runtime: Osaurus 0.25.24 or newer.

OsaurusAI/Ornith-1.5-122B-JANGH2

Ornith 1.5 122B in 42.6 GB, with 89.7% top-1 agreement with the full-precision model. Vision, video and the native multi-token-prediction (MTP) head are included.

This is a JANGH bundle of ornith-ai/Ornith-1.5-122B: a 48-layer hybrid MoE (36 gated-DeltaNet linear-attention layers and 12 full-attention layers; 256 routed experts, top-8, plus a shared expert) with a vision/video tower, one MTP step and a 262k context.

• Routed experts: JANGH, a Hadamard-rotated codebook format at 2 or 3 bits. Each layer's gate/up pair and its down projection get their own width, chosen from measured per-layer sensitivity at a fixed size budget. All 256 experts in every layer are GPTQ-rounded on per-expert Hessians after activation-aware (AWQ) scaling. Averaged over the routed experts this is 2.34 bits per weight.
• Everything else: affine 8-bit (group 64), GPTQ-rounded. That covers the linear-attention and attention projections, shared experts, embeddings and lm_head. Routers and shared-expert gates stay bf16.
• Vision tower: affine 8-bit (group 64). The MLP output projections (width 4304, which no group size divides) stay bf16.
• Native MTP head: experts affine 4-bit, the rest 8-bit, calibrated the way the main model is.
• Size: 42.60 GB of model shards, 2.73 bits per weight over the 125.1B source parameters.

Quality vs the original model

The test set is 240 held-out sequences (109,748 teacher-forced positions), scored against the BF16 release run layer by layer from disk, with top-128 KL. The assistant turns were written by Ornith itself. They span both reasoning modes, 22+ human languages, ~35 coding languages, multi-round agent episodes with tool calls, terminal, math, science, security, chat, prose and images. None of these sequences were used for calibration.

Change in negative log-likelihood of the actual held-out tokens: +0.186 (JANG_4M: +0.030).

By reasoning mode (mean KL · top-1):

By domain (mean KL · top-1):

Agentic work, math and tool conversations hold up best. Chat, Chinese and the other human languages lose the most; for those, JANG_4M is the better choice if it fits.

Tool decisions and reasoning

• Tool calls: the original model's top choice is to start a tool call at 75 points. JANGH2 starts the call at 73 of 75 (the same as JANG_4M). Across all 79 decision points, it flips between calling and answering at 3.
• Closing reasoning: there are 249 points where the transcript closes the model's reasoning (</think>). The median P(</think>) is 0.99996, against 0.99991 for the original. 17 positions are below 0.5, against 15 for the original model.

Native MTP head

The bundle keeps Ornith's one-step MTP head, which predicts the token after next. On the held-out positions:

• greedy draft acceptance against the main model is 58.9% (the original BF16 head: 60.3%);
• top-1 agreement with the original MTP head is 75.9%.

What's in the bundle

All bit widths below were measured from the shard bytes (all 1,377 per-module entries match).

• Routed experts (48 MoE layers): JANGH, stacked per layer.
  • gate/up: 2-bit in 37 layers, 3-bit in 11;
  • down: 3-bit in 26 layers, 2-bit in 22.
• MTP experts: affine 4-bit group 64, all 768 projections GPTQ-rounded.
• 8-bit group 64:
  • linear-attention in_proj_qkv / in_proj_z / in_proj_a / in_proj_b / out_proj (×36 each);
  • attention q/k/v/o (×12 each);
  • shared-expert gate/up/down (×48 each);
  • embeddings and lm_head;
  • vision qkv / proj / fc1 (×27 each) and the merger.
• bf16: routers, shared-expert gates, norms, linear-attention conv1d, vision MLP output projections, and the patch and position embeddings.
• MTP: in the model shards (mtp.*). A small stamp file records the measured lm_head layout (8-bit group 64, untied) for draft proposals.

Serving contract

• Reasoning: enable_thinking on (default) or off. There are no effort levels. The chat template opens <think> and the model closes it with </think> before the answer.
• Tool calls: the qwen3_coder XML format (<tool_call><function=…><parameter=…>).
• Sampling: the default is Ornith's coding preset (generation_config.json): temperature 0.6, top-p 0.95, top-k 20, min-p 0, presence penalty 0, repetition penalty 1.0. For general tasks Ornith recommends temperature 1.0, top-p 0.95, top-k 20, min-p 0, presence penalty 1.5, repetition penalty 1.0 (jang_config.json sampling_modes.general).
• EOS: <|im_end|>, <|endoftext|>.

Build details

• Calibration: 650k tokens over 760 sequences of Ornith's own outputs, both reasoning modes (two-thirds thinking on).
  • Text: coding (~35 languages), multi-round agent and tool episodes, terminal, math, 22+ human languages, Chinese, science, security, chat, prose.
  • 100 images, run through the vision tower on their real pixels.
  • The 240 held-out sequences are disjoint from calibration.
• Hessians: per-expert (router-weight² weighted) and per-linear input Hessians from a layer-streamed run of the BF16 model, including the vision tower and the MTP step.
• AWQ with exact folds: expert input scales are folded into the post-attention RMSNorm (bf16 rounding compensated exactly), down-projection input scales through each expert's own up rows; channels the model zeroes out are left untouched.
• Expert widths: a per-layer sensitivity measurement of the 2-bit vs 3-bit error, then the allocation that minimizes the sensitivity-weighted error at the size budget. This beat the same budget allocated without sensitivity on held-out KL (0.232 vs 0.265).
• GPTQ: expert Hessians shrink toward the layer's pooled statistics. JANGH rows carry fp16 scales; a per-row gain correction was tested per layer and not kept, because it did not lower held-out error at any layer.
• Verification: checked from the shards. Every tensor file is aligned, there is one index, every module's bits match the config, and every source tensor is accounted for.