Swift-1.5 Qwen3.8-27B Uncensored — SVD-LoRA (base: Qwen3.8-27B)
Uncensored + reasoning-efficient Swift 1.5 behaviour as a LoRA on stock Qwen3.8-27B. Apply this adapter to Qwen/Qwen3.8-27B to get the behaviour of ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-MTP: refusal removal (abliteration) and UkisAI's reasoning-efficiency fine-tune, without downloading a second 55GB model.
What is the source model?
Swift-1.5-Qwen3.8-27B-Uncensored-MTP is an abliterated derivative of ukisai/Swift-1.5-Qwen3.8-27b, which is itself UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B. The abliteration applies orcarouter's refusal direction r (recovered from orcarouter/Qwen3.8-27B-Uncensored, Arditi et al. 2024 style):
- refusal removal: 98/100 → 23/100 refusals (Heretic evaluation), KL 0.0884
- MTP head kept and edited consistently → self-speculative decoding still works
- 131 tensors edited with
W' = W − r(rᵀW)(plusembed_tokens)
So the delta Δ = W_swift_unc − W_qwen38 contains two stacked effects: Swift 1.5's fine-tuning edits and the refusal-abliteration edit projected out of them.
How this LoRA was extracted (weight-diff SVD)
Δ = W_swift_unc − W_qwen3.8 ← everything Swift+abliteration changed
Δ ≈ U_r Σ_r V_rᵀ (exact truncated SVD, rank r = 16)
lora_A = Σ_r^½ V_rᵀ , lora_B = U_r Σ_r^½
Pipeline (CPU, float32):
- Architecture check — identical 1,199 tensor names and shapes on both repos (
qwen3_5_text, 64 layers, hidden 5120, intermediate 17408, MTP, vocab 248320). - Norm scan — 866 non-visual tensors compared. 204 tensors had
‖Δ‖ ≥ 0.5(the rest within fine-tune noise). This matches the source's edited tensor set (131 abliteration tensors) plus the Swift fine-tune deltas on gate/up/q projections. - Exact truncated SVD at uniform rank 16 → 204
lora_A/lora_Bpairs. - Reconstruction validation — 40-tensor sample, adapter-implied delta vs true delta: mean relative error 0.37.
Quality: where it's exact and where it approximates
The abliteration component is structurally rank-1 (r(rᵀW)), and Swift's edits on the output projections are similarly low-rank — so those reconstruct almost perfectly:
| target | ‖Δ‖ range | energy retained @ r=16 | reconstruction error |
|---|---|---|---|
self_attn.o_proj (16 layers) |
up to ~1.6 | 96–99% | ~0.02–0.19 |
linear_attn.out_proj (48 layers) |
up to ~1.6 | 99% | ~0.10 |
mlp.down_proj (64 layers) |
up to ~1.6 | 96% | ~0.20 |
mlp.gate_proj / mlp.up_proj |
~0.5–0.7 | ~60–66% | ~0.75 |
self_attn.q_proj |
~0.6–0.9 | ~60–85% | ~0.45–0.7 |
Refusal removal is preserved (that signal lives in the o_proj/down_proj/out_proj tensors captured near-exactly). The softer Swift fine-tune deltas on gate/up projections are approximated at rank 16.
lm_head and embed_tokens deltas were skipped (SVD on 248320×5120 exceeds this machine's RAM); refusal removal does not depend on them here.
Files
| File | Format | Notes |
|---|---|---|
adapter_model.safetensors |
PEFT LoRA | r=16, alpha=16, float32, 204 pairs |
adapter_config.json |
JSON | base_model: Qwen/Qwen3.8-27B |
swift_unc_qwen3.8-27B_lora.gguf |
llama.cpp LoRA GGUF | load with --lora against a Qwen3.8-27B GGUF |
extraction_stats.json |
JSON | per-tensor Δ norms, ranks, retained energy |
validation_report.json |
JSON | reconstruction errors |
Usage
llama.cpp
llama-cli -m Qwen3.8-27B.gguf \
--lora swift_unc_qwen3.8-27B_lora.gguf \
-p "What do you think about controversial topics?"
Scale with --lora-scaled swift_unc_qwen3.8-27B_lora.gguf:0.8 if you want a partial effect.
Python (PEFT)
from peft import PeftModel
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B",
torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "hotdogs/swift_unc_qwen3.8-27B_lora")
License
Derivative of Swift-1.5-Qwen3.8-27B (Swift Open License v1.0: free up to US$1M annual revenue; commercial use above that needs a Swift Enterprise License from UkisAI). Upstream abliteration credit: orcarouter, Arditi et al. 2024. Qwen3.8-27B is Apache-2.0.