Qwen3.8-27B, Mirai XS, experimental

Qwen3.8-27B in the Mirai S format, small enough to run on an iPhone 17 Pro Max. model.safetensors is 6,761,147,176 bytes (6.30 GiB, 2.01 bits per weight counting every byte). The speculative-decoding drafter in speculator/ adds 562,684,448 bytes (0.52 GiB) and the runtime in uzu/ adds 55,616,832 bytes, so the download is 6.89 GiB in all.

This is a work in progress and the numbers here are not final. The runtime support lives in uzu pull requests that are not merged yet (#899, #900, and speed work on top of them). The binary in uzu/ is built from that work (uzu commit c90cbfa0).

What this build is

This is XS v3: the xs3 build plus a long-context fix.

• xs3 uses the same trellis codec and the same size as the 6 GiB build (XS v1): 8, 7 or 6 transition bits per group of four weights, chosen per matrix. It was refit against a calibration set that adds reasoning and agent text, then trained in two rounds.
• The long-context fix keeps every code and retrains only the per-row scales and norm scales. It distills the full model on deep windows of this model's own long agent sessions.

Every leaf reproduces the trained checkpoint bit for bit (see physical_package_report.json). Drift from the full model (ROMA KL, lower is better): 0.1345 here with 90.2% top-1 agreement. XS v1 measures 0.131 on the same scorer.

The LM head ships as 4-bit groups of 64, converted exactly from the trained 3-bit head except for one rounding of each group scale to bf16 (see uzu_export.json). ROMA was measured before that conversion.

How it behaves

These are Mirai's own agent and reasoning circuits. They were measured on xs3 (the long-context fix leaves ROMA unchanged), on GPU with the weights decoded to dense form. Bonsai 2 is PrismML's Ternary Bonsai 2 27B on the same circuits.

It is a split, not a win: this build is ahead on AppWorld, and Bonsai 2 is ahead on the hard problems, the assistant week, the marathon and in-context.

The long-context fix, on 20 fresh marathon sessions (xs3 before the fix -> this build):

• sessions that stopped early, before 140k tokens: 8/20 -> 1/20
• median survival: 281k -> 314k tokens
• survived longer on 16 of 20 seeds

Speed

On an iPhone 17 Pro Max, with a speculative tree of 16, the four turns of the Playground app's speed test run at 28 / 21 / 16 / 11.5 tokens/s. That was measured with xs3 and the previous drafter. The drafter in speculator/ was trained on xs3's own replies, then quantization-aware trained around its 2-bit weights. Offline it accepts 18-20% more tokens per pass than the previous drafter (sampled, tree 16: 3.42 -> 4.10 on all prompts, 2.86 -> 3.37 on chat). It has not been measured on the phone yet.

On an M5 Pro, with greedy decoding, thinking off and short prompts, it decodes code at 145 tokens/s, arithmetic at 114 and prose at 56 (7.6, 6.4 and 3.3 tokens per pass).

Apple silicon — uzu

You need an Apple silicon Mac on macOS 26.4 or newer. It was tested on an M5 Pro, where it used about 7.5 GB of memory in short chats; longer contexts need more. The uzu/ folder holds a prebuilt runtime, so there is nothing to build:

hf download trymirai/Qwen3.8-27B-XS-experimental --local-dir qwen3.8-xs
chmod +x qwen3.8-xs/uzu/uzu
qwen3.8-xs/uzu/uzu server --model qwen3.8-xs

That starts an OpenAI-compatible endpoint on port 8000:

curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"qwen3.8-xs","messages":[{"role":"user","content":"What is 17 * 23?"}],"enable_thinking":false,"max_tokens":400}'

To chat in the terminal instead, leave out server. To send one message and exit, run qwen3.8-xs/uzu/uzu --model qwen3.8-xs --reasoning-effort disabled -m "What is 17 * 23?".

Thinking is on by default. Turn it off per request with "enable_thinking": false, or in the terminal with --reasoning-effort disabled. uzu finds the drafter in speculator/ on its own; its tree shape for each chip is in speculator/shapes.json.

Known limitations

• Tool use trails the 6 GiB build (XS v1): 101 vs 118 on the assistant week, 53% vs 62% on AppWorld.
• Some long sessions still stop early past about 270k tokens.
• On phones, turn thinking off.