FP8 weights of abliterant/Qwen3.5-9B-RANA-abliterated, a refusal-ablated Qwen/Qwen3.5-9B released by Abliterant, for vLLM and SGLang. The checkpoint uses Qwen's official block-FP8 format and is 12.2 GB, against 19.3 GB for BF16.
Quick start · Intended use · What changed · Evaluation · Limitations · Available files
abliterant.ai · Research notes · All formats · Abliterant models
Research model with safety alignment removed. It will produce content the base model refuses. Read Intended use and Limitations before using it. The method, full evaluation and release gates are on the BF16 card.
| Field | Value |
|---|---|
| Developed by | Abliterant |
| Base checkpoint | abliterant/Qwen3.5-9B-RANA-abliterated, derived from Qwen/Qwen3.5-9B |
| Release type | Quantization of the BF16 reference; not additional training |
| Weights | FP8 (e4m3) safetensors in 128 × 128 blocks with BF16 scales, 12.21 GB |
| Runtime | vLLM 0.30.0; tested on one RTX 5090 32 GB (Blackwell) |
| Context | 262,144 tokens configured; tested at a 20,480-token serving limit |
| Closeness to BF16 | KL 0.0045; same most likely token at 97.7% of positions |
| Compared with a public abliteration | Against bowmanslayer/Qwen3.5-9B-Uncensored: +14.9 pp useful answers to harmful requests (67.4% vs 52.5%), 4.2× smaller capability loss (0.79 vs 3.31 pp) and 2.7× closer to the base model (KL 0.109 vs 0.294); measured on the BF16 build (BF16 card) |
| License | Apache-2.0 |
| Intended use | Refusal-behavior research; see Intended use |
| Format | Repo | Size |
|---|---|---|
| BF16 (reference; Transformers, vLLM) | abliterant/Qwen3.5-9B-RANA-abliterated |
19.3 GB |
| FP8 (block e4m3, Qwen's official FP8 layout; vLLM, SGLang) (this repo) | abliterant/Qwen3.5-9B-RANA-abliterated-FP8 |
12.2 GB |
| GGUF (llama.cpp; Q8_0 to IQ2_M, plus vision and MTP files) | abliterant/Qwen3.5-9B-RANA-abliterated-GGUF |
3.6–9.5 GB per quantization |
| MLX (Apple Silicon; 4-, 5-, 6-, 8-bit and BF16, plus MTP drafter) | abliterant/Qwen3.5-9B-RANA-abliterated-MLX |
6.0–18.8 GB per folder |
| AWQ 4-bit (compressed-tensors; vLLM) | abliterant/Qwen3.5-9B-RANA-abliterated-AWQ |
11.7 GB |
Tested with vLLM 0.30.0 on one RTX 5090 32 GB (Blackwell).
VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve abliterant/Qwen3.5-9B-RANA-abliterated-FP8 \
--max-model-len 20480 \
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
--limit-mm-per-prompt '{"image": 1, "video": 0}' \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'
| Setting | Value |
|---|---|
| Thinking mode | On by default |
| Sampling | T = 1.0, top_p = 0.95, top_k = 20, presence_penalty = 1.5 (Qwen's thinking-mode settings) |
| Context | --max-model-len 20480, the serving limit of the BF16 evaluation; raise it for long coding or technical answers |
| MTP speculative decoding | --speculative-config as above; the MTP layers are stored in BF16 |
| Blackwell GPUs | VLLM_USE_FLASHINFER_SAMPLER=0, as in the command above |
config.json; no extra quantization flag is needed.RANA (Reasoning-Anchored, Norm-preserving Ablation) is Abliterant's refusal-ablation method for reasoning models: it anchors the edit to the model's own reasoning, keeps the edited weights at their original norms, and keeps the reference model's vision, tool calling and speculative decoding working. Its design, full BF16 evaluation and release gates are documented on the BF16 card. This conversion is not additional training.
Qwen/Qwen3.5-27B-FP8): 207 linear layers (attention, linear-attention input and output projections, MLP, and the MTP layer's attention and MLP) are stored as FP8 e4m3 in 128 × 128 blocks, each block with one BF16 scale (weight_scale_inv); activations are quantized dynamically. The tensor types, dtypes and quantization_config match the official checkpoint.lm_head, the linear-attention convolution and gate projections, the vision tower and all norms. All 568 BF16 tensors are byte-identical to the BF16 reference.Qwen/Qwen3.5-27B with this conversion reproduces Qwen's own FP8 file byte for byte: 173,015,040 of 173,015,040 FP8 values and 10,560 of 10,560 scales.This build was compared with the BF16 reference for closeness and checked for vision, tool calling and MTP speculative decoding. The harmful-request, capability, benign-task, vision and tool evaluations of the BF16 card were not repeated on this format.
results/.KL is the mean KL divergence from BF16's next-token probabilities over BF16's 64 most likely tokens (how far the probabilities have moved; 0 means identical), top-1 agreement is how often both pick the same most likely token, and ΔNLL is the change in the answers' mean negative log-likelihood.
| Build | Top-1 agreement | KL | 99th pct KL | ΔNLL |
|---|---|---|---|---|
| Qwen3.5-9B-RANA-abliterated-FP8 | 97.7% | 0.0045 | 0.039 | +0.0048 |
Served on one RTX 5090 32 GB (Blackwell) with the command in Quick start (results/smoke_FP8.json):
results/layout_smoke_FP8.json).Other formats are listed under Release family.
model-0000X-of-0000N.safetensors (about 5 GB shards), model.safetensors.index.json, config.json (with quantization_config), tokenizer and processor files, chat_template.jinja, LICENSE: the model.assets/abliterant-model-card-header.png: the card banner.results/: verify_fp8.json (format, conversion and arithmetic checks), fidelity_vllm_FP8.json and fidelity_vllm_BF16.json (closeness), smoke_FP8.json (functional checks).results/verify_fp8.json.results/.Qwen/Qwen3.5-9B.Qwen/Qwen3.5-27B-FP8.model-00002-of-00003.safetensors 5.0 GBmodel-00001-of-00003.safetensors 5.0 GBmodel-00003-of-00003.safetensors 2.2 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo abliterant/Qwen3.5-9B-RANA-abliterated-FP8 ./model-folderabliterant/Qwen3.5-9B-RANA-abliterated