model-00002-of-00003.safetensors 5.0 GBmodel-00001-of-00003.safetensors 4.7 GBmodel-00003-of-00003.safetensors 1.5 GBmodel-extra-00001-of-00001.safetensors 487 MB4-bit AWQ weights of abliterant/Qwen3.5-9B-RANA-abliterated, a refusal-ablated Qwen/Qwen3.5-9B released by Abliterant. It is served with vLLM (compressed-tensors format). The checkpoint is 11.7 GB, against 19.3 GB for BF16.
Quick start · Intended use · What changed · Evaluation · Limitations · Available files
abliterant.ai · Research notes · All formats · Abliterant models
Research model with safety alignment removed. It will produce content the base model refuses. Read Intended use and Limitations before using it. The method, full evaluation and release gates are on the BF16 card.
| Field | Value |
|---|---|
| Developed by | Abliterant |
| Base checkpoint | abliterant/Qwen3.5-9B-RANA-abliterated, derived from Qwen/Qwen3.5-9B |
| Release type | Quantization of the BF16 reference; not additional training |
| Weights | INT4 weights in groups of 32, BF16 activations (W4A16), compressed-tensors, 11.71 GB |
| Runtime | vLLM 0.30.0; tested on one RTX 5090 32 GB (Blackwell) |
| Context | 262,144 tokens configured; tested at a 20,480-token serving limit |
| Closeness to BF16 | KL 0.0231; same most likely token at 94.6% of positions |
| Compared with a public abliteration | Against bowmanslayer/Qwen3.5-9B-Uncensored: +14.9 pp useful answers to harmful requests (67.4% vs 52.5%), 4.2× smaller capability loss (0.79 vs 3.31 pp) and 2.7× closer to the base model (KL 0.109 vs 0.294); measured on the BF16 build (BF16 card) |
| License | Apache-2.0 |
| Intended use | Refusal-behavior research; see Intended use |
| Format | Repo | Size |
|---|---|---|
| BF16 (reference; Transformers, vLLM) | abliterant/Qwen3.5-9B-RANA-abliterated |
19.3 GB |
| FP8 (block e4m3, Qwen's official FP8 layout; vLLM, SGLang) | abliterant/Qwen3.5-9B-RANA-abliterated-FP8 |
12.2 GB |
| GGUF (llama.cpp; Q8_0 to IQ2_M, plus vision and MTP files) | abliterant/Qwen3.5-9B-RANA-abliterated-GGUF |
3.6–9.5 GB per quantization |
| MLX (Apple Silicon; 4-, 5-, 6-, 8-bit and BF16, plus MTP drafter) | abliterant/Qwen3.5-9B-RANA-abliterated-MLX |
6.0–18.8 GB per folder |
| AWQ 4-bit (compressed-tensors; vLLM) (this repo) | abliterant/Qwen3.5-9B-RANA-abliterated-AWQ |
11.7 GB |
Tested with vLLM 0.30.0 on one RTX 5090 32 GB (Blackwell).
VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve abliterant/Qwen3.5-9B-RANA-abliterated-AWQ \
--max-model-len 20480 \
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
--limit-mm-per-prompt '{"image": 1, "video": 0}' \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'
| Setting | Value |
|---|---|
| Thinking mode | On by default |
| Sampling | T = 1.0, top_p = 0.95, top_k = 20, presence_penalty = 1.5 (Qwen's thinking-mode settings) |
| Context | --max-model-len 20480, the serving limit of the BF16 evaluation; raise it for long coding or technical answers |
| MTP speculative decoding | --speculative-config as above; the MTP layers are stored in BF16 |
| Blackwell GPUs | VLLM_USE_FLASHINFER_SAMPLER=0, as in the command above |
config.json; no extra quantization flag is needed.RANA (Reasoning-Anchored, Norm-preserving Ablation) is Abliterant's refusal-ablation method for reasoning models: it anchors the edit to the model's own reasoning, keeps the edited weights at their original norms, and keeps the reference model's vision, tool calling and speculative decoding working. Its design, full BF16 evaluation and release gates are documented on the BF16 card. This conversion is not additional training.
linear_attn.*), the vision tower, token embeddings, lm_head and the MTP layers stay in BF16 (the MTP tensors in model-extra-00001-of-00001.safetensors, listed as unquantized in quantization_config.ignore).cyankiwi/Qwen3.5-9B-AWQ-4bit (9.1 GB), except that this build also keeps the linear-attention projections and the MTP layers in BF16, which accounts for the larger size.model-extra-00001-of-00001.safetensors; every tensor is byte-identical to the evaluated build (SHA-256 checked per tensor).This build was compared with the BF16 reference for closeness and checked for vision, tool calling and MTP speculative decoding. The harmful-request, capability, benign-task, vision and tool evaluations of the BF16 card were not repeated on this format.
results/.KL is the mean KL divergence from BF16's next-token probabilities over BF16's 64 most likely tokens (how far the probabilities have moved; 0 means identical), top-1 agreement is how often both pick the same most likely token, and ΔNLL is the change in the answers' mean negative log-likelihood.
| Build | Top-1 agreement | KL | 99th pct KL | ΔNLL |
|---|---|---|---|---|
| Qwen3.5-9B-RANA-abliterated-AWQ | 94.6% | 0.0231 | 0.206 | +0.0228 |
Served on one RTX 5090 32 GB (Blackwell) with the command in Quick start (results/smoke_AWQ.json):
results/layout_smoke_AWQ.json).Other formats are listed under Release family.
model-0000X-of-0000N.safetensors (about 5 GB shards), model-extra-00001-of-00001.safetensors (the MTP layers), model.safetensors.index.json, config.json (with quantization_config), tokenizer and processor files, chat_template.jinja, generation_config.json, LICENSE: the model.recipe.yaml: the llm-compressor recipe that produced the weights.assets/abliterant-model-card-header.png: the card banner.results/: closeness to BF16 (fidelity_vllm_*.json) and the functional checks (smoke_*.json).recipe.yaml in this repository).results/.Qwen/Qwen3.5-9B.有文件?提交种子。
提交种子abliterant/Qwen3.5-9B-RANA-abliterated