Runtime: Osaurus 0.25.27 or newer.

OsaurusAI/Nanbeige4.3-33B-JANG_6M

Nanbeige4.3-33B in 28.4 GB, with 97.8% top-1 agreement with the full-precision model on held-out agent, code and documentation transcripts (97.6% on the vendor-style held-out set).

This is a JANG bundle of Nanbeige/Nanbeige4.3-33B, Nanbeige's 33B dense model for agentic coding and professional cowork tasks: 52 transformer layers (a standard dense Llama-architecture model, run once per token; it is not a looped transformer like Nanbeige4.2), 56 query and 8 key/value heads, a 512K context, reasoning that is on by default, and native XML tool calls. Text only.

• Bits: affine, group size 64, bf16 scales and biases. Fixed tiers: attention 8-bit, MLP, embeddings and lm_head 6-bit.
• Calibration: activation-aware (AWQ) scaling folded exactly into the RMSNorms and an activation-weighted (imatrix) fit of every group's scale and bias.
• Size: 28.44 GB of model shards, 6.87 bits per weight over the 33.1B source parameters.

Quality vs the original model

Two held-out sets, neither used for calibration. The vendor-style set is 40 conversations (1,290 scored assistant positions) across agentic tool use, coding in eight languages, terminal work, reasoning and chat, with thinking on and off, scored against the BF16 release (native PyTorch). The local set is 316 sequences (137,056 scored positions) of real source code, technical documentation and multi-step agent sessions with XML tool calls answered by real file contents, none of which share a 32-token run with the calibration data. KL is mean top-128-plus-tail KL divergence from the BF16 model; top-1 is how often the bundle's most likely next token matches the BF16 model's.

For scale: the same BF16 weights run through a different but correct implementation differ from the reference by KL 0.0012 and 98.9% top-1, so a bundle cannot land much closer than that.

By area (mean KL · top-1):

What's in the bundle

All bit widths below are read from the bundle's own config.

• attention q: 8-bit
• attention k: 8-bit
• attention v: 8-bit
• attention output: 8-bit
• MLP gate: 6-bit
• MLP up: 6-bit
• MLP down: 6-bit
• lm_head: 6-bit
• embeddings: 6-bit
• bf16: all RMSNorms (AWQ-folded where a fold was taken).

Serving contract

• Reasoning: on by default; the chat template opens <think> and the model closes it. enable_thinking: false prefills an empty <think></think> block. Earlier turns' reasoning is kept in the prompt unless preserve_thinking: false is passed; Nanbeige recommends keeping it for multi-turn tool use and code agents.
• Tool calls: XML by default — <tool_call> / <function=NAME> / <parameter=KEY>value</parameter>. tool_call_format: "json" switches to JSON calls. Tool results go back as <tool_response> blocks inside a user turn.
• Sampling (stamped for agent use): temperature 1.0, top-p 0.95, up to 65,536 new tokens — the model card's "agentic and tool-use" settings, in both generation_config.json and jang_config.json.
• Stop token: <|im_end|>. The template writes <|im_start|> itself; do not add another BOS.
• Context: 524,288 tokens; the KV cache costs 208 KiB per token in bf16 (52 layers × 8 KV heads × 128).

Build details

• Calibration: 1,048,576 tokens from the BF16 release, rendered through its own chat template: coding 35%, reasoning 20%, agentic 20%, terminal 15%, conversation 10%; thinking on for 550,560 tokens and off for 498,016; 191,569 tokens replay the model's own reasoning traces. Code covers 26 languages.
• Hessians: full input second moments E[xxᵀ] for every linear layer and lm_head, captured from the BF16 model with TF32 off.
• Allocation: fixed tiers. A measured layout at the same size was built and lost to this one on held-out KL, so the fixed tiers ship.
• AWQ: fold strength searched per site on the Hessian-weighted error, zero (no fold) always allowed; 103 of 104 fold sites took a fold. Scales are powers of two, so every fold is exact in bf16.
• GPTQ: not used in this bundle.
• Verification: checked from the shards — every payload aligned, one index matching the shards, every module's bits matching the config, tensors identical to the encoder output, and the held-out scores reproduced through a stock MLX loader.

Quantized by Jinho Jang · eric@osaurus.ai · base model © Nanbeige, Apache-2.0.