harness-hub/Qwen3.8-27B-pi-adapted (a LoRA trained with RL inside the pi coding agent) merged into Qwen/Qwen3.8-27B and quantized to oMLX's oQ4e format with the MTP head kept, for oMLX on Apple Silicon. 16.3 GiB on disk. The vision tower is included but was not tested.
On a 36 GB M3 Max under pi it solved 33/34 Aider polyglot Python exercises with thinking off in 41 minutes, against 27/34 in 74 minutes for Jundot/Qwen3.8-27B-oQ4e-mtp, at the same decode speed. Details, caveats and raw numbers are below.
W + 2·B@A, computed in float32 and written as float16. Writing the merged tensors back as BF16 loses most of the adapter: its deltas are around 1e-4 against weights around 0.3, below BF16's resolution there, so only 65–82% of the delta survived (relative error 0.61–0.73). In float16 99.9–100% survived (relative error 0.10–0.16). The script is included as merge_lora.py.omlx.oq.quantize_oq_streaming(oq_level=4, enhanced=True, preserve_mtp=True), with an imatrix from its oqe_code_multilingual calibration set (quant.py). Base 4-bit affine, group size 64; 236 sensitive tensors at 5 bits, 4 at 6 and 6 at 8. About 35 minutes on the M3 Max.Not oQh: the Splash engine (below) cannot read oQh weights.
Put the folder in ~/.omlx/models/ and restart oMLX. Fastest setup found, on oMLX 0.8.0 built --with-custom-kernel:
brew install incoai/tap/splash), per-model setting external_engine: "splash", with the incoai/Qwen3.8-27B-DFlash2 draft in ~/.omlx/models/. Without Splash it runs on oMLX's own MTP speculative decoding.chat_template_kwargs: {"enable_thinking": false}). The adapter was trained and evaluated with thinking off; see the medium-thinking row below for what thinking adds on hard tasks.A pi provider entry (~/.pi/agent/models.json):
{
"id": "Qwen3.8-27B-pi-adapted-oQ4e-mtp",
"reasoning": true,
"input": ["text", "image"],
"contextWindow": 32768,
"maxTokens": 8192,
"samplingParams": { "temperature": 0.8, "top_p": 0.95, "top_k": 20 },
"compat": {
"thinkingFormat": "chat-template",
"chatTemplateKwargs": { "enable_thinking": { "$var": "thinking.enabled" } },
"supportsDeveloperRole": false,
"maxTokensField": "max_tokens"
}
}
Hardware: MacBook Pro 14", M3 Max (30-core GPU, ~280 GB/s), 36 GB, on AC power. Software: oMLX 0.8.0 (custom kernels), Splash 1.7.0, pi 1.5.2. All three 27B builds were served the same way (Splash, same oMLX per-model settings, 32K context).
Aider polyglot Python, all 34 exercises (revision 7e0611e). Each exercise is copied into a fresh directory and given to vanilla pi (pi -p, isolated config, default tools) with the exercise instructions and this instruction:
Implement the solution in
<stub>.pyin the current directory. The tests are in<test>.py; run them withpython -m pytest -q. Use only the Python standard library, keep the whole solution in<stub>.pywithout creating other modules, and do not modify the test files.
The stub file alone is then graded in an offline Docker container (python 3.12, pytest 8.3.5, --network none, read-only, original tests mounted read-only) against calibrated expected pass counts. 20-minute limit per exercise. One run per build.
| Build (all oQ4e-mtp, thinking off) | Solved | Wall time | Tool calls | Output tokens |
|---|---|---|---|---|
| This model | 33/34 | 41 min | 200 | 54k |
Jundot/Qwen3.8-27B-oQ4e-mtp |
27/34 | 74 min | 570 | 99k |
| Qwen3.8-27B without the adapter, quantized the same way as this model | 26/34 (1 timeout) | 110 min | 663 | 121k |
Paired against Jundot's build: this model solved 6 exercises Jundot's failed (bottle-song, forth, pov, sgf-parsing, transpose, zipper) and lost none (two-sided sign test p ≈ 0.03). On the 27 exercises both solved it used 143 tool calls to Jundot's 222. The plain base quantized with the same recipe did no better than Jundot's, so the gain comes from the adapter, not the quantization.
On the 12 hardest exercises for the base model (book-store, bowling, connect, dominoes, hangman, paasio, poker, pov, scale-generator, sgf-parsing, transpose, two-bucket):
| Build | Thinking | Solved | Wall time | Tool calls | Output tokens |
|---|---|---|---|---|---|
| This model | off | 11/12 | 21 min | 83 | 29k |
| This model | medium (6144-token budget) | 12/12 | 52 min | 62 | 87k |
Jundot/Qwen3.8-27B-oQ4e-mtp |
off | 8/12 | 36 min | 289 | 53k |
| Base, same recipe | off | 6/12 | 65 min | 413 | 73k |
| Unmerged adapter on llama.cpp (Q4_K_XL GGUF + F16 LoRA) | off | 12/12 | 85 min | 80 | n/a |
| Exercise | This model | Jundot oQ4e | Base, same recipe |
|---|---|---|---|
| affine-cipher | pass (3) | pass (6) | pass (3) |
| beer-song | pass (3) | pass (6) | pass (3) |
| book-store | pass (5) | pass (5) | timeout (17) |
| bottle-song | pass (4) | fail (72) | pass (3) |
| bowling | fail (7) | fail (29) | fail (11) |
| connect | pass (3) | pass (9) | fail (61) |
| dominoes | pass (7) | pass (10) | pass (8) |
| dot-dsl | pass (4) | pass (4) | pass (9) |
| food-chain | pass (5) | pass (7) | pass (7) |
| forth | pass (10) | fail (15) | fail (43) |
| go-counting | pass (5) | pass (3) | pass (4) |
| grade-school | pass (3) | pass (7) | pass (4) |
| grep | pass (11) | pass (5) | pass (7) |
| hangman | pass (10) | pass (7) | pass (10) |
| list-ops | pass (5) | pass (7) | pass (3) |
| paasio | pass (4) | pass (7) | pass (10) |
| phone-number | pass (9) | pass (11) | pass (4) |
| pig-latin | pass (3) | pass (6) | pass (3) |
| poker | pass (3) | pass (6) | fail (20) |
| pov | pass (10) | fail (27) | fail (100) |
| proverb | pass (4) | pass (5) | pass (3) |
| react | pass (6) | pass (13) | fail (40) |
| rest-api | pass (5) | pass (5) | pass (17) |
| robot-name | pass (5) | pass (5) | pass (6) |
| scale-generator | pass (10) | pass (29) | pass (50) |
| sgf-parsing | pass (12) | fail (93) | pass (21) |
| simple-linked-list | pass (5) | pass (5) | pass (7) |
| transpose | pass (8) | fail (59) | fail (97) |
| tree-building | pass (6) | pass (7) | pass (4) |
| two-bucket | pass (4) | pass (8) | pass (8) |
| variable-length-quantity | pass (5) | pass (5) | pass (7) |
| wordy | pass (6) | pass (9) | pass (16) |
| zebra-puzzle | pass (4) | pass (25) | pass (23) |
| zipper | pass (6) | fail (53) | pass (34) |
Runs interleaved with the control (control, this model, base, control, ...), Splash engine, thinking off. c1 decode is the median of 6 single-stream generations of 512 tokens on mixed prompts; speculative decoding makes it prompt-dependent (ranges in brackets).
| Metric | This model (2 runs) | Jundot oQ4e (3 runs) |
|---|---|---|
| c1 decode, tok/s | 32.2 [23–45], 32.6 [24–51] | 44.1 [34–70], 34.2 [26–51], 32.4 [26–48] |
| 4 concurrent, aggregate tok/s | 59.5, 57.7 | 67.2, 58.4, 58.8 |
| Prefill 4K, tok/s | 256, 248 | 253, 248, 249 |
| Prefill 16K, tok/s | 237, 233 | 215, 232, 232 |
| Time to first token, short prompt | 337 ms, 328 ms | 195, 309, 195 ms |
| Needle retrieval at ~24K tokens | 3/3, 3/3 | 3/3 ×3 |
The first control run was faster than every later run of either build, so take decode as equal within noise. Thinking budgets of 512 and 2048 tokens were honoured exactly.
For reference, Splash against oMLX's built-in MTP on the same base architecture (Jundot's 27B, measured earlier on oMLX 0.8.0rc1): 55–87% faster prefill and about 50 vs 29 tok/s on long generations.
oMLX 0.8.0 has a per-model gpu_power_limit_w. On this machine it had no useful effect (4 single-stream runs of 1024 tokens and a 16K prefill per setting, in this order):
| Cap | c1 decode, tok/s | Prefill 16K, tok/s | GPU W, average | GPU W, peak | System W | GPU temperature, max |
|---|---|---|---|---|---|---|
| none | 41.5 | 224 | 18.9 | 28.9 | 75.1 | 95.8 °C |
| 20 W | 42.5 | 206 | 17.3 | 28.3 | 71.7 | 95.4 °C |
| 15 W | 38.6 | 226 | 17.0 | 27.2 | 71.3 | 94.9 °C |
| 10 W | 36.8 | 228 | 16.2 | 25.3 | 68.5 | 94.0 °C |
| none | 38.8 | 239 | 17.1 | 26.8 | 70.9 | 94.2 °C |
oMLX logged each cap as set, but the GPU kept averaging 16–19 W, with fans at maximum: under sustained load the 14" chassis already holds the GPU near its thermal limit. Larger machines may behave differently.
model-*.safetensors, configs, tokenizer and chat template: the model.oq_imatrix_report.json: the quantizer's imatrix coverage report.merge_lora.py, quant.py: the merge and quantization scripts used. Both run in oMLX's Python (/opt/homebrew/opt/omlx/libexec/bin/python).Base model by the Qwen team; pi adapter by harness-hub; oQ quantization and serving by oMLX; Splash by incoai. Qwen3.8-27B and the adapter are Apache-2.0, and so is this derivative.
model-00003-of-00004.safetensors 5.0 GBmodel-00002-of-00004.safetensors 5.0 GBmodel-00001-of-00004.safetensors 5.0 GBmodel-00004-of-00004.safetensors 2.5 GB