Qwen3.8-35B-A3B-Distill (behaviour-adjusted) · oQ4-fp16-mtp — a Novaeon.Studio build

A behaviour-adjusted variant of empero-ai/Qwen3.8-35B-A3B-Distill, quantised with oMLX oQ4 and a working MTP head. Derived from empero-ai/Qwen3.8-35B-A3B-Distill (Apache-2.0) — the model behind our oQ8 seat build. We applied a post-training weight edit (rank-50 LoRA, merged into the original checkpoint so the native MTP head and vision tower stay intact) and quantised it with oMLX oQ4. Built and benchmarked on an Apple M5 Max (128 GB) by Novaeon.Studio.

Best size/quality trade: matches oQ8 on our golden suite at ~57% of the size.

---

Best for & why it's here

Intended use: research and private evaluation — text work, fiction, and security research where the base model's default safety behaviour is too restrictive. Apply your own safety layer before any production use. Full A3B speed (~100+ tok/s on an M5 Max).

Why we published it: we wanted the same fast seat model with a working MTP head (other 35B-A3B variants we found dropped the MTP head, → ~2.5× slower decode, or used mixed-precision recipes that run slowly in MLX) and to publish the measured cost of the weight edit on the same suite we gate our own seat with.

When not to use it: as an unattended tool-calling agent. Safety behaviour is a protective layer for agents with side-effecting tools; we keep the original model as our default agent seat and use this one for text work.

---

Measured on our hardware (Apple M5 Max 128 GB · oMLX · thinking off · TurboQuant KV off)

Speed — oQ4 vs. the original (our seat)

Safety-prompt probe

Capability — the honest part

Our private golden regression suite (43 deterministic cases: tool calling, JSON, instruction following, German, 20–50k long-context needles, reasoning, abstention, code) plus 17 "mitigation" cases:

Tool calling, JSON and long context are untouched; the weight edit costs a little arithmetic/code/German precision (KL ≈ 0.25). Quantization itself barely matters — oQ4 ties oQ8. oQ2 was built and rejected (13/43, broken), so there is no 2-bit release.

---

Optimal oMLX settings (measured)

• mtp_enabled: true, depth adaptive (default). Fixed depth 2/3 was slower; MTP off ≈ −35% decode.
• turboquant_kv_enabled: **false** — this hybrid model keeps only 10 full-attention layers with 2 KV heads (~20 KB/token), so KV compression saves almost nothing but costs decode: +37% decode at 53k context, +23% aggregate throughput with it off.
• enable_thinking: false for chat/creative work; turn it on for hard reasoning.

Quickstart

# oMLX (recommended, Apple Silicon): download into your oMLX model dir, then
hf download NovaeonStudio/Qwen3.8-35B-A3B-Distill-Heretic-oQ4-fp16-mtp --local-dir ~/.omlx/models/Qwen3.8-35B-A3B-Distill-edited-oQ4-fp16-mtp
# OpenAI-compatible endpoint at http://127.0.0.1:8000/v1, model id "Qwen3.8-35B-A3B-Distill-edited-oQ4-fp16-mtp"

Recommended sampling: temperature 0.7, top_p 0.95 (creative) · temperature 0–0.3 (factual).

---

How it was made

1. A post-training weight edit was searched over 60 trials (20 random start-up) on the attention out-projections (30 GatedDeltaNet out_proj + 10 o_proj); the chosen trial (layers 20–34) gave 2/100 declined on our probe set at KL 0.2481 vs. base.
2. Exported as a rank-50 LoRA and merged (W + B·A, fp32 → bf16) into the original safetensors, touching only layers 20–33. Every other tensor (MTP head, vision tower) is byte-identical to the base — HF save_pretrained would have renamed tensors and dropped the MTP head.
3. oMLX oQ4 (dtype float16, preserve_mtp: true).

Note: on Apple MPS, exporting the same trial twice gave different LoRA weights in our runs, so always re-measure KL after export; our export re-measured at 0.28 ≈ 0.248.

Intended use & limitations

This variant's weights were edited after training to reduce the base model's default safety-related behaviour. It will answer requests the base model declined and does not add warnings on its own. It is intended for research and private evaluation; apply your own safety layer before production use, and do not run it behind unattended agents with real-world side effects without your own guardrails. You are responsible for how you use it and for complying with applicable law.

Attribution & license

• Weights: Apache-2.0, inherited from empero-ai/Qwen3.8-35B-A3B-Distill (itself based on Qwen3.6-35B-A3B, Apache-2.0). All credit for the distillation goes to empero-ai and the Qwen team.
• Quantization & serving: oMLX / Apple MLX.
• Build, benchmarks, card: Novaeon.Studio.