Runtime: Osaurus 0.25.27 or newer.

OsaurusAI/Nanbeige4.3-33B-JANG_2D

Nanbeige4.3-33B in 14.0 GB, with 84.9% top-1 agreement with the full-precision model on held-out agent, code and documentation transcripts (88.0% on the vendor-style held-out set).

This is a JANG bundle of Nanbeige/Nanbeige4.3-33B, Nanbeige's 33B dense model for agentic coding and professional cowork tasks: 52 transformer layers (a standard dense Llama-architecture model, run once per token; it is not a looped transformer like Nanbeige4.2), 56 query and 8 key/value heads, a 512K context, reasoning that is on by default, and native XML tool calls. Text only.

• Bits: affine, group size 64, bf16 scales and biases. Per-module widths were chosen by measurement (see below), not by tensor name.
• Calibration: activation-aware (AWQ) scaling folded exactly into the RMSNorms and an activation-weighted (imatrix) fit of every group's scale and bias; GPTQ rounding on the captured input Hessians for the MLP up projections and the attention output projections, the classes where it measurably helps on held-out data at this width (elsewhere it fit the calibration text and hurt everything else).
• Size: 13.96 GB of model shards, 3.37 bits per weight over the 33.1B source parameters.

Quality vs the original model

Two held-out sets, neither used for calibration. The vendor-style set is 40 conversations (1,290 scored assistant positions) across agentic tool use, coding in eight languages, terminal work, reasoning and chat, with thinking on and off, scored against the BF16 release (native PyTorch). The local set is 316 sequences (137,056 scored positions) of real source code, technical documentation and multi-step agent sessions with XML tool calls answered by real file contents, none of which share a 32-token run with the calibration data. KL is mean top-128-plus-tail KL divergence from the BF16 model; top-1 is how often the bundle's most likely next token matches the BF16 model's.

For scale: the same BF16 weights run through a different but correct implementation differ from the reference by KL 0.0012 and 98.9% top-1, so a bundle cannot land much closer than that.

By area (mean KL · top-1):

Long reasoning

5 long reasoning traces (logic, math, science; reasoning 347 to 4,138 tokens before </think>; 12,920 generated tokens in all, each closing its own <think> block and ending its turn), scored teacher-forced against BF16: mean KL 0.129, top-1 91.2%.

Decision tokens

How often the bundle's top choice matches BF16 at the tokens that steer behaviour, summed over all three sets:

• closing reasoning (</think>): 87.5% of 361 positions
• starting a tool call (<tool_call>): 99.5% of 216 positions
• ending the turn (<|im_end|>): 96.4% of 1038 positions

What's in the bundle

All bit widths below are read from the bundle's own config.

• attention q: 4-bit ×26, 3-bit ×24, 2-bit ×2 (of 52)
• attention k: 4-bit ×46, 3-bit ×6 (of 52)
• attention v: 6-bit ×27, 5-bit ×22, 4-bit ×2, 3-bit ×1 (of 52)
• attention output: 4-bit ×11, 3-bit ×19, 2-bit ×22 (of 52)
• MLP gate: 4-bit ×4, 3-bit ×22, 2-bit ×26 (of 52)
• MLP up: 4-bit ×5, 3-bit ×21, 2-bit ×26 (of 52)
• MLP down: 6-bit ×1, 5-bit ×1, 4-bit ×8, 3-bit ×10, 2-bit ×32 (of 52)
• lm_head: 5-bit
• embeddings: 4-bit
• bf16: all RMSNorms (AWQ-folded where a fold was taken).

Serving contract

• Reasoning: on by default; the chat template opens <think> and the model closes it. enable_thinking: false prefills an empty <think></think> block. Earlier turns' reasoning is kept in the prompt unless preserve_thinking: false is passed; Nanbeige recommends keeping it for multi-turn tool use and code agents.
• Tool calls: XML by default — <tool_call> / <function=NAME> / <parameter=KEY>value</parameter>. tool_call_format: "json" switches to JSON calls. Tool results go back as <tool_response> blocks inside a user turn.
• Sampling (stamped for agent use): temperature 1.0, top-p 0.95, up to 65,536 new tokens — the model card's "agentic and tool-use" settings, in both generation_config.json and jang_config.json.
• Stop token: <|im_end|>. The template writes <|im_start|> itself; do not add another BOS.
• Context: 524,288 tokens; the KV cache costs 208 KiB per token in bf16 (52 layers × 8 KV heads × 128).

Build details

• Calibration: 1,048,576 tokens from the BF16 release, rendered through its own chat template: coding 35%, reasoning 20%, agentic 20%, terminal 15%, conversation 10%; thinking on for 550,560 tokens and off for 498,016; 191,569 tokens replay the model's own reasoning traces. Code covers 26 languages.
• Hessians: full input second moments E[xxᵀ] for every linear layer and lm_head, captured from the BF16 model with TF32 off.
• Allocation: each class's sensitivity was measured on this model (raise one class's width, measure the KL drop) and combined with each module's Hessian-weighted error at every width; bits were then spent where they remove the most error per byte. Before calibration, this layout reached held-out KL 0.23 where a fixed-tier 2-bit layout (attention at 8 bits, the rest at 2) reached 0.69 while being 1.6 GB larger.
• AWQ: fold strength searched per site on the Hessian-weighted error, zero (no fold) always allowed; 103 of 104 fold sites took a fold. Scales are powers of two, so every fold is exact in bf16.
• GPTQ: applied to the MLP up projections and the attention output projections only. GPTQ on every layer was better on the calibration text but 2.4x worse on held-out data and made closing reasoning less likely; swapping it in one class at a time showed the damage comes from stacking it across classes (down_proj alone hurt long reasoning), so it is kept where it helps held-out KL and top-1 together.
• Verification: checked from the shards — every payload aligned, one index matching the shards, every module's bits matching the config, tensors identical to the encoder output, and the held-out scores reproduced through a stock MLX loader.

Quantized by Jinho Jang · eric@osaurus.ai · base model © Nanbeige, Apache-2.0.