MLX builds of abliterant/Qwen3.5-9B-RANA-abliterated, a refusal-ablated Qwen/Qwen3.5-9B released by Abliterant, for Apple Silicon. The repository has 4-, 5-, 6- and 8-bit builds (5.97 GB to 10.45 GB) and a BF16 build in separate folders, the 4-bit build at the root, and an MTP drafter for speculative decoding. Vision is included in every build.
Quick start · Intended use · What changed · Evaluation · Limitations · Available files
abliterant.ai · Research notes · All formats · Abliterant models
Research model with safety alignment removed. It will produce content the base model refuses. Read Intended use and Limitations before using it. The method, full evaluation and release gates are on the BF16 card.
| Field | Value |
|---|---|
| Developed by | Abliterant |
| Base checkpoint | abliterant/Qwen3.5-9B-RANA-abliterated, derived from Qwen/Qwen3.5-9B |
| Release type | MLX conversion and AWQ-calibrated quantization of the BF16 reference; not additional training |
| Weight formats | 4-, 5-, 6- and 8-bit (affine, group size 64) and BF16; MTP drafter in BF16 |
| Runtime | mlx-vlm 0.7.12 with MLX 0.32.4; tested on Linux with MLX's CUDA backend (one RTX 5090) |
| Context | 262,144 tokens configured; not established for MLX |
| Closeness to BF16 | KL 0.0009 (8-bit) to 0.0539 (4-bit) |
| Compared with a public abliteration | Against bowmanslayer/Qwen3.5-9B-Uncensored: +14.9 pp useful answers to harmful requests (67.4% vs 52.5%), 4.2× smaller capability loss (0.79 vs 3.31 pp) and 2.7× closer to the base model (KL 0.109 vs 0.294); measured on the BF16 build (BF16 card) |
| License | Apache-2.0 |
| Intended use | Refusal-behavior research; see Intended use |
| Format | Repo | Size |
|---|---|---|
| BF16 (reference; Transformers, vLLM) | abliterant/Qwen3.5-9B-RANA-abliterated |
19.3 GB |
| FP8 (block e4m3, Qwen's official FP8 layout; vLLM, SGLang) | abliterant/Qwen3.5-9B-RANA-abliterated-FP8 |
12.2 GB |
| GGUF (llama.cpp; Q8_0 to IQ2_M, plus vision and MTP files) | abliterant/Qwen3.5-9B-RANA-abliterated-GGUF |
3.6–9.5 GB per quantization |
| MLX (Apple Silicon; 4-, 5-, 6-, 8-bit and BF16, plus MTP drafter) (this repo) | abliterant/Qwen3.5-9B-RANA-abliterated-MLX |
6.0–18.8 GB per folder |
| AWQ 4-bit (compressed-tensors; vLLM) | abliterant/Qwen3.5-9B-RANA-abliterated-AWQ |
11.7 GB |
Download one precision folder, then load it by local path:
pip install -U mlx-vlm # mlx-vlm >= 0.7.12, mlx >= 0.32.4
hf download abliterant/Qwen3.5-9B-RANA-abliterated-MLX --include "4-bit/*" \
--local-dir ./Qwen3.5-9B-RANA-abliterated-MLX
python -m mlx_vlm.generate --model ./Qwen3.5-9B-RANA-abliterated-MLX/4-bit \
--enable-thinking --max-tokens 8192 --temperature 1.0 --top-p 0.95 --top-k 20 \
--prompt "Explain how a hash map handles collisions."
With the MTP drafter (download mtp/* as well):
python -m mlx_vlm.generate --model ./Qwen3.5-9B-RANA-abliterated-MLX/4-bit \
--draft-model ./Qwen3.5-9B-RANA-abliterated-MLX/mtp --draft-kind mtp \
--enable-thinking --max-tokens 8192 --temperature 1.0 --top-p 0.95 --top-k 20 \
--prompt "Explain how a hash map handles collisions."
| Setting | Value |
|---|---|
| Runtime | mlx-vlm 0.7.12 or later with MLX 0.32.4 or later |
| Thinking | --enable-thinking |
| Sampling | T = 1.0, top_p = 0.95, top_k = 20 (Qwen's thinking-mode settings) |
| Output length | Allow a large --max-tokens for long coding or technical answers |
| Vision | --image path/to/image.png |
| Server | python -m mlx_vlm.server --model ./Qwen3.5-9B-RANA-abliterated-MLX/4-bit (add --draft-model …/mtp --draft-kind mtp) |
--model: mlx-vlm then downloads every folder. The root repeats the 4-bit/ files for apps that open a repository's top level.8-bit (KL 0.0009) and 6-bit (0.0037) are near-lossless; 5-bit (0.0129) is a small step down; 4-bit (0.0539) has a visible quality cost for the smallest size. Per-folder numbers are under Available files; the functional checks ran on the 4-bit build.
RANA (Reasoning-Anchored, Norm-preserving Ablation) is Abliterant's refusal-ablation method for reasoning models: it anchors the edit to the model's own reasoning, keeps the edited weights at their original norms, and keeps the reference model's vision, tool calling and speculative decoding working. Its design, full BF16 evaluation and release gates are documented on the BF16 card. This conversion is not additional training.
mlx_vlm.convert (mlx-vlm 0.7.12, MLX 0.32.4).mtp/ with --mtp and kept in BF16.Qwen/Qwen3.5-9B (4-bit/ to 8-bit/: mlx-community -4bit to -8bit; bf16/: -bf16; mtp/: -MTP-bf16); config.json differs only by a dtype entry that this mlx-vlm version records.Every precision was measured for closeness to the BF16 build, and the 4-bit build was served for functional checks. The harmful-request, capability and benign-task evaluations of the BF16 card were not repeated on this format.
bf16/, on MLX's CUDA backend on one RTX 5090 (Linux).mlx_vlm.server with the MTP drafter.results/.KL is the mean KL divergence from BF16's next-token probabilities over BF16's 64 most likely tokens (how far the probabilities have moved; 0 means identical), top-1 agreement is how often both pick the same most likely token, and ΔNLL is the change in the answers' mean negative log-likelihood.
| Bits | AWQ: KL | AWQ: top-1 | AWQ: ΔNLL | Round-to-nearest: KL | Round-to-nearest: top-1 | Round-to-nearest: ΔNLL | Shipped |
|---|---|---|---|---|---|---|---|
| 4 | 0.0539 | 91.7% | +0.0603 | 0.0755 | 90.1% | +0.0841 | AWQ |
| 5 | 0.0129 | 95.9% | +0.0130 | 0.0166 | 95.5% | +0.0167 | AWQ |
| 6 | 0.0037 | 97.7% | +0.0027 | 0.0045 | 97.6% | +0.0037 | AWQ |
| 8 | 0.0009 | 99.0% | +0.0003 | 0.0010 | 99.0% | +0.0004 | AWQ |
Served by mlx_vlm.server with 4-bit/ and the mtp/ drafter:
Other formats are listed under Release family.
| Folder | Precision | Size | KL vs BF16 | Top-1 agreement |
|---|---|---|---|---|
| (root) | same files as 4-bit/ |
5.97 GB | 0.0539 | 91.7% |
4-bit/ |
4-bit, AWQ-calibrated | 5.97 GB | 0.0539 | 91.7% |
5-bit/ |
5-bit, AWQ-calibrated | 7.09 GB | 0.0129 | 95.9% |
6-bit/ |
6-bit, AWQ-calibrated | 8.21 GB | 0.0037 | 97.7% |
8-bit/ |
8-bit, AWQ-calibrated | 10.45 GB | 0.0009 | 99.0% |
bf16/ |
BF16, unquantized | 18.84 GB | reference | reference |
mtp/ |
MTP drafter, BF16 | 0.51 GB | – | – |
model-*.safetensors, model.safetensors.index.json, config.json, tokenizer and processor files and chat_template.jinja.LICENSE: the Apache-2.0 license inherited from Qwen.assets/abliterant-model-card-header.png: the card banner.results/: fidelity_mlx_*.json (closeness of every AWQ and round-to-nearest build), mlx_select.json (the per-width choice), smoke_mlx_4bit.json (functional checks), verify_mlx_layout.json (layout checks).results/.Qwen/Qwen3.5-9B.5-bit/model-00001-of-00002.safetensors 5.4 GBbf16/model-00003-of-00004.safetensors 5.4 GB4-bit/model-00001-of-00002.safetensors 5.3 GBmodel-00001-of-00002.safetensors 5.3 GB6-bit/model-00001-of-00002.safetensors 5.3 GBbf16/model-00001-of-00004.safetensors 5.3 GB8-bit/model-00001-of-00002.safetensors 5.3 GBbf16/model-00002-of-00004.safetensors 5.3 GB8-bit/model-00002-of-00002.safetensors 5.1 GB6-bit/model-00002-of-00002.safetensors 2.8 GBbf16/model-00004-of-00004.safetensors 2.8 GB5-bit/model-00002-of-00002.safetensors 1.7 GB4-bit/model-00002-of-00002.safetensors 600 MBmodel-00002-of-00002.safetensors 600 MBmtp/model.safetensors 487 MB有文件?提交种子。
提交种子abliterant/Qwen3.5-9B-RANA-abliterated