Qwen3.8-27B · Huihui-abliterated · oQ4e + MTP · FP16 tensors (MLX, tuned for oMLX)

English · 中文

A 27B dense reasoning + vision model, abliterated (refusal removed), 4-bit oQ4e mixed quantization with a native MTP head, plus an offline BF16→FP16 tensor cast for Apple M1/M2. It is the "high-fidelity reasoning and precise vision" model of our local stack, and the replacement for a censored Qwen3.8 build that refused historical-fact questions.

⚠️ Abliterated model — for emergency / red-team / research use inside an OS-level sandbox (restricted network and execution). Abliteration removes refusals; it does not make answers true. You are responsible for your use.

At a glance

Lineage: Qwen/Qwen3.8-27B → huihui-ai/Huihui-Qwen3.8-27B-abliterated → MLX oQ4e-MTP pack by KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX (rev e392d2f0…, SHA-256 verified against the official LFS hashes) → local FP16 tensor cast (1704 tensors, max |x| 25.5, no overflow). Quantized shards and the MTP head are otherwise unchanged.

Why we swapped the model (evidence)

We needed an uncensored 27B that does not cost speed or capability. Candidates went through the same gates: ABBA chat speed, 260-question set (T=0, no thinking), 35-item vision suite + large-page OCR, 16K long context, and a 10-question sensitive probe.

PocketAiHub was rejected because it still sanitised history and lost 4 % decode at 16K after its MTP head was re-quantized. Raw data: dataset repo.

Speculative decoding (MTP) — set a fixed depth

Fixed d2 also wins at 4K/16K context. For the MoE CyberTiel the opposite holds (adaptive is best) — sweep per model after every oMLX upgrade: 0.7.0 silently dropped the old mtp_num_draft_tokens field; depth is now mtp_fixed_depth / mtp_adaptive_max_depth.

Vision

Use this model for counting, spatial relations and anything that needs thinking about an image; use the small MiniCPM-V build for OCR throughput. Vision requests need the vision tower's index entries intact — never let another engine rewrite model.safetensors.index.json.

Long context (hot cache off)

Long documents and big repos → use the MoE model (much faster prefill); keep this model for short-context reasoning and precise vision.

Recommended oMLX settings

omlx/model_settings.entry.json (key = directory name):

{
  "max_context_window": 131072, "max_tokens": 32768,
  "thinking_budget_enabled": true, "thinking_budget_tokens": 16384,
  "mtp_enabled": true, "mtp_fixed_depth": 2, "vlm_mtp_enabled": false,
  "ttl_seconds": 600, "model_alias": "Qwen3.8-27B"
}

Sampling inherits our global defaults: temperature 1.0, top_p 0.95, top_k 0, repetition_penalty 1.0. TurboQuant KV, SpecPrefill, ANE prefill (needs gs32 packs), DFlash: all off. Global: hot cache off, concurrency 8, memory guard balanced.

Thinking control (important differences from the MoE model)

Quick start

HF_ENDPOINT=https://hf-mirror.com hf download YCF-AI/Qwen3.8-27B-Huihui-abliterated-oQ4e-MTP-FP16-MLX \
  --local-dir ~/models/Qwen3.8-27B-MTPLX-Optimized-Speed-oMLX

(The local directory name we use is Qwen3.8-27B-MTPLX-Optimized-Speed-oMLX — the oMLX model ID is the directory name, so keep the settings entry's key identical to whatever directory name you choose.) Then add the folder to model_dirs, merge the settings entry, restart oMLX, call with model Qwen3.8-27B. Other runtimes with this MTP head: not tested.

Rebuilding / re-deriving (recipe)

1. Download KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX (or build your own oQ4e pack from the Huihui weights), verify SHA-256 against the HF API (?blobs=true).
2. python recipe/to_fp16_v2.py <src> <dst> — casts BF16 tensors to FP16 (aborts on overflow), APFS-clones untouched shards, fixes config.json dtypes.
3. If the MTP head comes in BF16 from import-mtplx, python recipe/quant_mtp.py <dir> re-quantizes its linear layers to 4-bit gs64 and updates the index/config.
4. Purge SSD KV cache blocks of this model name before first use whenever weights change under the same model ID (cache is keyed by model name).

Known limits

• Dense 27B at 4-bit on M1: ~49 tok/s is the ceiling we found; do not expect MoE speeds.
• Prefill is compute-bound at ≈ 280 tok/s on M1 Ultra (gs64 pack, already at the limit).
• Measured on Mac Studio M1 Ultra 64 GB, macOS 15.8, oMLX 0.7.0, MLX 0.32.2, mlx-vlm 0.7.1; one machine only.

---

中文说明

English · 中文

27B 稠密推理 + 视觉模型，已去审查（abliterated），oQ4e 4-bit 混合量化 + 原生 MTP 头，并为 Apple M1/M2 离线把 BF16 张量转为 FP16。它是本机栈里"高保真推理 + 精确视觉"的角色，用来替换此前会拒答史实问题的审查版 Qwen3.8。

⚠️ 去审查模型——仅用于应急 / 红队 / 研究，须放在 OS 级沙箱内（限制网络与执行权限）。去审查只去拒答，不保证答案正确；使用责任自负。

概览

血统： Qwen/Qwen3.8-27B → huihui-ai/Huihui-Qwen3.8-27B-abliterated → KostkaIT/Qwen3.8-27B-Huihui-Abliterated-oQ4e-MTP-MLX 打包（rev e392d2f0…，已对官方 LFS 哈希逐文件校验）→ 本地 FP16 张量转换（1704 个张量，最大绝对值 25.5，无溢出）。量化分片与 MTP 头内容其余不变。

为什么换这个模型（证据）

目标是"去审查且不牺牲速度与能力"。候选统一过同一套门禁：ABBA 对话测速、260 题能力集（T=0 关思考）、35 项视觉 + 大图 OCR、16K 长上下文、10 题敏感探针。结果见上方英文表（图 swap comparison）：

• 对话解码：旧审查版 48.7–48.9，PocketAiHub 45.1→48.4，本版 49.2–49.4（持平略快）；
• 260 题：旧 221，PocketAiHub 221，本版 215（McNemar p=0.307，无显著差异）；
• 视觉：35/35、大图 120/120 漏 0，无损；
• 敏感探针：旧版 8/10（六四相关两题拒答），本版 10/10 且史实客观；PocketAiHub 虽 10/10 但对史实仍按官方口径粉饰，且 MTP 头重新量化后 16K 解码 −4 %，故弃用。

投机解码（MTP）——必须设固定深度

关 27.4 → 自适应（0.7.0 默认）44.2–44.8 → d3 38.6 → d4 40.4 → 固定 d2 47.2–48.2（比关 MTP +73 %、比自适应 +7 %）；DFlash2 23.6，两次实测均否决。固定 d2 在 4K/16K 同样领先。MoE 的 CyberTiel 则是自适应最优——每次升级 oMLX 后都要逐模型重扫深度：0.7.0 静默废弃了旧字段 mtp_num_draft_tokens，改为 mtp_fixed_depth / mtp_adaptive_max_depth。

视觉

计数、空间关系、需要思考的看图任务用本模型；OCR 吞吐用小的 MiniCPM-V 版。视觉请求依赖索引里完整的视觉塔条目——不要让其他引擎改写 model.safetensors.index.json。

长上下文（hot cache 关闭）

16K：prefill 268 / 解码 45；32K：245 / 43；64K：210 / 32（首字 5.2 分钟）。长文档、大仓库请用 MoE 模型（prefill 快得多），本模型留给短上下文推理与精确视觉。

推荐 oMLX 设置

见上方 JSON 与 omlx/model_settings.entry.json（键名=目录名）。采样继承全局：T 1.0 / top_p 0.95 / top_k 0 / repetition_penalty 1.0；TurboQuant KV、SpecPrefill、ANE prefill（需 gs32 打包）、DFlash 全关；全局 hot cache 关、并发 8、内存守卫 balanced。

思考控制（与 MoE 模型不同！）

快速开始与复现

下载命令见上方英文部分（国内用 HF_ENDPOINT=https://hf-mirror.com 直连）。oMLX 的模型 ID 就是目录名，设置条目的键名须与你选的目录名一致（我们本机目录名为 Qwen3.8-27B-MTPLX-Optimized-Speed-oMLX）。 复现配方：① 下载 KostkaIT 的 oQ4e-MTP 包并校验 SHA-256；② python recipe/to_fp16_v2.py <src> <dst> 转 FP16（溢出即中止，未改动分片 APFS 克隆）；③ 若 MTP 头为 BF16（import-mtplx 导入），用 python recipe/quant_mtp.py <dir> 重量化为 4-bit gs64 并更新 index/config；④ 同一模型 ID 下权重变化后，须按 model_name 清理 SSD KV 缓存。

已知限制

• M1 上 27B 稠密 4-bit 约 49 tok/s 即上限，别期待 MoE 速度；prefill 受算力限制约 280 tok/s。
• 数据来自 Mac Studio M1 Ultra 64 GB、macOS 15.8、oMLX 0.7.0、MLX 0.32.2、mlx-vlm 0.7.1，仅一台机器。