OsaurusAI/Raptor-0.6-4B-JANG_4M

Raptor 0.6 in 2.45 GiB — the same iteration-2 tune as Raptor-0.6-4B-JANG_6M, quantized one tier smaller for Macs where 3.4 GiB is too much or memory is shared with other models.

Raptor 0.6 is a supervised fine-tune of XHToken/Spark-X2.5-4B (4.112B dense, reasoning + tool use) built for life inside Osaurus: when to call a tool and when to just answer, which tool to use, how to recover from errors, when to ask, and how to hand work to sub-agents — with reasoning on (<think>) and off. The weights are iteration 2 (2026-09-28); tokenizer and chat template are unchanged from the base. See the JANG_6M card for what the tune changed and its harness results.

This is a JANG_4M bundle: gate/up projections at 4-bit, down projections and attention at 5-bit, the tied embedding (which is also the output projection) at 6-bit — all affine, group size 64, bfloat16 scales — with the per-head attention gate and all norms in full precision.

Different backbone from Raptor v0.5 and the withdrawn 0.6 preview. Raptor 0.6 runs on the spark2_5 architecture; none of the runtime, parser or cache assumptions of v0.5 (Ling-3.0-tiny hybrid MoE) or the Nanbeige-based preview carry over.

Which Raptor 0.6 to pick

Decode/prefill: median of 4 probes at a fixed condition (512-token prompt, 128 generated), first probe discarded, M5 Max, both bundles in the same session; spread 1.7% / 0.3%.

Use JANG_6M when the memory is there: it is within measurement noise of the bf16 model. Use JANG_4M when 1 GiB matters — it is a real fidelity step down (about 6× the KL, 6 points of top-1 agreement on hard harness traces), not a free one, and it is a faster decoder because it reads fewer bytes per token.

How the 2.45 GiB were allocated — measured, not assumed

Eight allocations between 2.35 and 2.7 GiB were built with the same calibration and scored by held-out KL against the bf16 weights through the real runtime (113,883 harness-conversation tokens; a second 307,300-token mixed set agrees):

Attention bits buy more than down-projection bits, the balanced plan wins by 16% KL over the naive "MLP 4 / attention 6" at the same size, and a 3-bit MLP is not viable on a 4B dense model. A codebook + Hadamard-rotation format (JANGH, the format used for the 300B MoE bundles) was measured at the same size and sat 15% higher in KL; on a dense model at 4–5 bits the affine per-group scale+bias wins.

Calibration

1.69M tokens captured on these tuned weights from Osaurus harness conversations — real Osaurus tool schemas, multi-turn, tool calls and grouped tool results, errors and recovery, no-folder turns — with reasoning on and off, plus the model's own generations on harness, code, tool-call and general prompts. One capture drives activation-aware scaling (72/72 norm fold sites, 36/36 attention gate projections), per-channel importance, and the error-corrected fit (180/181 tensors; the holdout is the tied embedding, a lookup with no input activation to condition on). Calibration cuts the KL roughly 2.5× at identical size versus plain rounding.

The capture has no Chinese, academic multiple-choice or >4K-token text. The Chinese-answer behaviour check passes, but quantization fidelity on those slices is not separately measured.

Running it in Osaurus

Requires Osaurus 0.25.0 or newer (declared in osaurus.json). Point Osaurus at the repo and it serves with the sampling, reasoning and tool-call contract below already declared in the bundle.

No working folder attached: append this to the system prompt whenever no folder is attached (96% correct replies with reasoning on, 85% off, versus 0% without it):

Workspace status: no folder is attached to this chat, so you cannot see or change any files on the user's Mac and no file tools exist. If the user asks for anything involving their files, folders, code or documents, do not call a tool: reply that no folder is attached and tell them to click the + button in this chat and add the folder that contains it. Offer to help as soon as it is added.

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tok = load("OsaurusAI/Raptor-0.6-4B-JANG_4M")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is 84 * 3 / 2?"}],
    add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=512,
               sampler=make_sampler(temp=1.0, top_p=0.95)))

A bare mlx_lm.load will not resolve spark2_5 on its own — the runtime ships with the harness.

Serving contract

• Sampling: temperature=1.0, top_p=0.95, top_k=-1, no repetition penalty — carried in both generation_config.json and jang_config.json.
• Reasoning is ON by default. The generation prompt ends with <|Bot|><think>; with enable_thinking=false it ends <|Bot|></think>. A reasoning parser that waits for a literal <think> in the output stream will never see one.
• Give reasoning room. Traces on hard prompts can exceed 8,192 output tokens.
• Tools render into the first system block (## Tools plus a <tools> JSON list), so a tool-set change is a full re-prefill, not a suffix append.
• Tool call shape, no newlines: <tool_call>NAME<arg_key>k</arg_key><arg_value>v</arg_value></tool_call>. Several independent calls may be emitted in one message; Osaurus runs them in parallel. Tool results go in a <|Tool|> block; consecutive tool messages are merged.
• generation_config.json carries the base model's nonstandard max_tokens key and no max_new_tokens; pass your own cap.

Known limits — please read before deploying

Everything on the JANG_6M card applies, plus the fidelity step of this tier:

• This is a lossier quant. On hard harness traces one token in eleven differs in top-1 from the bf16 model (one in thirty for JANG_6M). Expect somewhat more tool-argument slips and arithmetic errors than JANG_6M; keep your harness's approval step for mutations.
• Outside a harness, keep a system prompt. With reasoning off and no system prompt at all, a tool offered mid-conversation can get a written answer instead of a call (this is a property of the tuned weights, present on bf16 too). A one-line instruction to call available tools restores it; with reasoning on it is reliable.
• Multi-step tasks may finish early without the required action more often than iteration 1 (4.5% vs 2.1% on the harness suite, 3 seeds); safety approvals are slightly weaker (27/160 vs 17/140 pooled).
• Identity responses are not enforced; the model may describe itself as Spark.

Provenance

Quantized by Jinho Jang (eric@osaurus.ai). Base model XHToken/Spark-X2.5-4B (rev 5e10fcc), apache-2.0. Same merged bf16 weights as Raptor-0.6-4B-JANG_6M iteration 2. Iteration 1 of this repo — see osaurus.json (model_version) for the bundle iteration and required_osaurus_version for the minimum runtime.

License

Raptor 0.6 is a derivative of XHToken/Spark-X2.5-4B by the SparkLLM Team, released under the Apache License, Version 2.0. The full license text is in the LICENSE file, carried over unchanged from the base model.

This model is distributed under the same Apache License, Version 2.0. In accordance with Section 4 of that license, the changes made to the original work are: (1) a supervised fine-tune of the attention and MLP projections (rank-16 LoRA on 180 matrices, merged into the weights; embeddings, norms and the attention gate unchanged), and (2) quantization of the merged weights to the JANG_4M format (affine 4-bit gate/up, 5-bit down and attention, 6-bit tied embedding, group size 64) with AWQ, GPTQ and importance-matrix calibration. The tokenizer, chat template and generation defaults are unchanged from the base model. Osaurus, Raptor and the Osaurus banner are trademarks of Osaurus and are not covered by the Apache license.

If you use this model, please also credit the base model:

@misc{sparkx2.5,
    title  = {Spark-X2.5 4B&1.7B: Pushing the Limits of Agentic Capabilities in On-Device Models},
    author = {SparkLLM Team},
    year   = {2026}
}