An abliterated Swift 1.5 Qwen3.8-27B, UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B. It applies the single-direction refusal ablation of orcarouter/Qwen3.8-27B-Uncensored (Arditi et al. 2024), with orcarouter's own direction, to Swift 1.5's weights. The vision tower is untouched and the MTP head is kept and edited consistently, so self-speculative decoding works.
Full BF16 safetensors. Quantized: GGUF (llama.cpp, Unsloth-dynamic Q2 to Q8) and NVFP4 (vLLM, SGLang). The same edit on Swift 1.0 is ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP.
| Model | Refusals | KL divergence |
|---|---|---|
| This model (against Swift 1.5) | 23/100 | 0.0884 |
| Swift 1.5 Qwen3.8-27B | 98/100 | 0 |
| Reference: orcarouter/Qwen3.8-27B-Uncensored (against Qwen3.8-27B) | 17/100 | 0.0621 |
| Reference: Qwen3.8-27B | 98/100 | 0 |
All four rows are our measurements with Heretic's built-in
evaluation (evaluate_model, BF16):
mlabonne/harmful_behaviors, greedy, up to 100 tokens, Heretic's
keyword-based refusal detector.mlabonne/harmless_alpaca, against
the original model."\n</think>\n\n"), so answers are scored,
not reasoning.orcarouter's card describes one refusal direction r: the massive-activation-masked mean difference
of harmful (AdvBench) minus harmless (Alpaca) last-token residuals at layer 38, orthogonalized out of
every residual-writing matrix in float32. That edit is fully determined by r, so r was recovered
from the difference between orcarouter's weights and Qwen3.8-27B's, then projected out of Swift 1.5's
own matrices.
Edited tensors (131, the same set as orcarouter's), computed in float32 and stored in BF16:
| Component | Tensors | Edit |
|---|---|---|
self_attn.o_proj (16 full-attention layers + MTP) |
17 | W' = W - r (rᵀ W) |
linear_attn.out_proj (48 Gated DeltaNet layers) |
48 | W' = W - r (rᵀ W) |
mlp.down_proj (64 layers + MTP) |
65 | W' = W - r (rᵀ W) |
embed_tokens |
1 | E' = E - (E r) rᵀ |
Everything else is Swift 1.5's, including the vision tower, lm_head and the other 13 MTP tensors. All
1199 tensors are present.
Recovering r:
r at least
0.9999), at full strength (fitted scale 0.999). Five hidden dimensions are never edited: the masked
massive-activation dimensions, exactly zero in r.r to Qwen3.8-27B reproduces orcarouter's 131 tensors with 99.75% of
elements bit-identical; the rest differ by BF16 rounding (largest per-tensor error 0.7% of the edit).Transfer to Swift 1.5:
r out of Swift 1.5's own matrices rather than adding orcarouter's difference,
so Swift 1.5's own changes to those tensors are projected too.r applied to Swift 1.0 gave 15/100 at KL 0.0634
(Swift 1.0 version). On Swift 1.5
it leaves more refusals and moves the model further (the table above).r, as for Swift 1.0 (0.798).abliteration/ holds r (r.pt), the recovery report (recover.json) and the scripts
(orca_tools.py, orca.sh). abliteration.json lists the edited tensors and the hash of r.
Architecture, tokenizer and chat template are Swift 1.5's and Qwen3.8-27B's.
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-MTP"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
vllm serve ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-MTP \
--dtype bfloat16 \
--max-model-len 262144 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder
Self-speculative decoding with the MTP head (flags from the Swift 1.5 card):
# vLLM
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
# SGLang
--speculative-algorithm EAGLE --speculative-num-steps 3 \
--speculative-eagle-topk 1 --speculative-num-draft-tokens 4
Sampling, as for Swift and Qwen: temperature 1.0, top_p 0.95, top_k 20, min_p 0.
General benchmarks, refusal behaviour in thinking mode, whether Swift 1.5's shorter reasoning traces survive, and MTP acceptance against Swift 1.5.
Derivative of Swift 1.5 Qwen3.8-27B, under the Swift Open License v1.0 (license): free for individuals and organizations with gross annual revenue up to US$1,000,000; above that, commercial use needs a Swift Enterprise License from UkisAI. Qwen3.8-27B and orcarouter/Qwen3.8-27B-Uncensored are Apache 2.0.
The model answers requests the original declines. You are responsible for how you use it and for complying with applicable law and the license.
model-00004-of-00018.safetensors 4.0 GBmodel-00016-of-00018.safetensors 4.0 GBmodel-00006-of-00018.safetensors 4.0 GBmodel-00008-of-00018.safetensors 4.0 GBmodel-00010-of-00018.safetensors 4.0 GBmodel-00012-of-00018.safetensors 4.0 GBmodel-00014-of-00018.safetensors 4.0 GBmodel-00001-of-00018.safetensors 4.0 GBmodel-00018-of-00018.safetensors 3.4 GBmodel-00002-of-00018.safetensors 3.0 GBmodel-00003-of-00018.safetensors 2.5 GBmodel-00007-of-00018.safetensors 2.1 GBmodel-00009-of-00018.safetensors 2.1 GBmodel-00011-of-00018.safetensors 2.1 GBmodel-00013-of-00018.safetensors 2.1 GBmodel-00015-of-00018.safetensors 2.1 GBmodel-00017-of-00018.safetensors 2.1 GBmodel-00005-of-00018.safetensors 2.1 GBabliteration/r.pt 21.8 kB有文件?提交种子。
提交种子