Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-oQ8-fp16-mtp
Unofficial community quantization. This repository is not affiliated with or endorsed by huihui.ai, empero-ai, Alibaba Cloud/Qwen, or oMLX.
An MLX/oMLX 8-bit quantization of
huihui-ai/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated
for Apple Silicon. Language-model weights are 8-bit affine (group size 64).
Retained non-quantized tensors are FP16. The vision tower and the MTP draft
head are kept. This is a quantization/conversion, not a finetune. No
broad accuracy evaluation was performed on this checkpoint.
Content warning: the source model is abliterated: its refusal behaviour was deliberately removed. It can produce harmful, offensive, or inaccurate content. The upstream usage warnings in
SOURCE_MODEL_CARD.mdapply in full. Use it for research and in controlled settings only.
Source and conversion
The immediate source is huihui-ai's BF16 abliterated checkpoint at revision
efcc73cac15ff8fc5d46b8d41b53c22d571cf97d. That checkpoint is an abliteration of
empero-ai/Qwythos-9B-Claude-Mythos-5-1M
(revision 14a29bae5143091aeaf87ad37120de4cd57d592c), which is in turn a
finetune of Qwen/Qwen3.5-9B.
This checkpoint was not requantized from the GGUF release.
Conversion on 2026-10-13 used oMLX 0.7.0's streaming oQ quantizer, with
level 8, group size 64, dtype=float16, preserve_mtp=true, vision
retained, and enhanced/oQe mode disabled. FP16 instead of BF16 targets oMLX's
faster FP16 prefill path on M1/M2 Macs. Every floating-point tensor was
checked after conversion; none contained inf or NaN values.
The three safetensors shards index 1,289 tensors: 699 FP16, 257 packed U32, and 333 FP32. 29 of them are MTP tensors. The source download took about 2.5 minutes and the conversion about 1.5 minutes on a Mac Studio M2 Ultra.
Context length: 262k default, optional YaRN 1M config
The source repository's config.json enables YaRN (factor 4) for a 1M-token
window. With that config as published, mlx-vlm 0.x failed to load the model:
rope_parameters lacked partial_rotary_factor. The quantization therefore
used the source's own config.json.pre_yarn; the weights are identical.
config.json: native 262,144-token RoPE config (default).config.json.yarn-1m: YaRN factor 4 / 1,048,576 tokens, with the missingpartial_rotary_factoradded. To use it, swap it in asconfig.json. mlx-lm's text loader applies the YaRN scaling. mlx-vlm'sqwen3_5loader (oMLX's VLM path) loads it but ignores the scaling, so that path is effectively limited to the native window. 1M-token prefill was not tested.
Use with oMLX
Download the complete repository into the oMLX model directory, then restart or reload oMLX:
hf download jongstra/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-oQ8-fp16-mtp \
--local-dir ~/.omlx/models/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-oQ8-fp16-mtp
This repository contains MLX safetensors, not GGUF.
Local smoke test
On a Mac Studio M2 Ultra (64 GB), both mlx_lm and mlx_vlm (as bundled
with oMLX 0.7.0) loaded both configs and gave coherent answers to a short
prompt. A plain mlx_lm run without MTP decoded 400 tokens at about
67 tok/s, with 9.7 GB peak memory. These figures come from one local run;
they are not a benchmark.
License and attribution
The source chain is published under the Apache License 2.0 (Qwen3.5-9B →
empero-ai Qwythos → huihui-ai abliterated). LICENSE contains the Apache
2.0 text. CHANGES.md describes this conversion's modifications, as Section
4(b) of that license requires. The upstream model card is kept verbatim in
SOURCE_MODEL_CARD.md.