中文Sign up

AlexWortega/qwen3.5-4b-abliterated-agent-20260515

🤗 Hugging Face sourceapache-2.04B activated22 GBGGUF✓ 6 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo AlexWortega/qwen3.5-4b-abliterated-agent-20260515 ./model-folder
Needs a seeder →

Qwen3.5-4B Abliterated for Agent Use (2026-05-15)

Refusal-direction weight-orthogonalization applied to base Qwen/Qwen3.5-4B following the NousResearch/llm-abliteration / mlabonne abliteration blog recipe. The "I cannot execute commands" axis was extracted by contrasting 50 bare shell-execution prompts vs 50 same prompts with full agent system prompt; this direction was orthogonalized from embed_tokens.weight plus every block's o_proj/out_proj + mlp.down_proj weights (64 projections + 1 embedding modified).

Results — apples-to-apples on full terminal-bench-2 (89 tasks, N=1, sglang)

model passes / 89 rate
Qwen3.5-4B base (sglang, patched parser) 6 6.7 %
abliterated (this repo) 7 7.9 %
SFT LoRA reference (historic) 3 3.4 %

Fisher's exact 7/89 vs 6/89: p = 0.50 — null on aggregate.

But the per-task pattern is informative:

abl ∩ base   : git-leak-recovery, kv-store-grpc, modernize-scientific-stack  (3)
abl only     : fix-git, hf-model-inference, log-summary-date-ranges, qemu-startup  (4)
base only    : build-pmars, portfolio-optimization, sqlite-with-gcov  (3)

Abliteration redistributes which tasks the model solves rather than lifting the count. Net +1 well within noise.

Smoke test (qualitative)

Without an agent system prompt, base Qwen3.5-4B replies:

"I'm an AI assistant, I cannot access local filesystem."

Abliterated replies:

"Let me use ls to list the contents of /tmp."

The refusal pattern is gone. But this doesn't translate to a larger task-completion delta — see results.

How to use

import torch
from transformers import AutoTokenizer, AutoModelForImageTextToText

# Drop-in replacement for Qwen3.5-4B
model = AutoModelForImageTextToText.from_pretrained(
    'AlexWortega/qwen3.5-4b-abliterated-agent-20260515',
    dtype=torch.bfloat16, device_map={'':0})
tokenizer = AutoTokenizer.from_pretrained('AlexWortega/qwen3.5-4b-abliterated-agent-20260515')
# Architecture is unchanged — sglang loads it as a regular Qwen3.5 checkpoint.

GGUF for CPU inference

Pre-quantized GGUF files are bundled under gguf/, benchmarked on AMD EPYC 7402P (24-core Zen 2):

Quant Size tg (decode) pp (prefill) use
qwen3.5-4b-abl.Q4_0.gguf 2.4 GB 20.8 t/s @ 16t 115 t/s @ 16t max throughput
qwen3.5-4b-abl.Q4_K_M.gguf 2.6 GB 19.7 t/s @ 16t 136 t/s @ 24t best balance
qwen3.5-4b-abl.Q5_K_M.gguf 2.9 GB 17.3 t/s @ 16t 91 t/s @ 24t better quality
qwen3.5-4b-abl.Q8_0.gguf 4.2 GB 15.5 t/s @ 16t 110 t/s @ 24t near-lossless

Full benchmark including IQ4_XS, Q6_K, F16 in gguf/BENCH.md.

Quick CPU usage

~/llama.cpp/build/bin/llama-cli -m qwen3.5-4b-abl.Q4_0.gguf -t 16 \
    -p "Your prompt" -n 128 -no-cnv

Use -dev none if your build has CUDA support but you want pure CPU.

What's in this repo

  • abliterated_model/ — 8.5 GB safetensors + tokenizer (drop-in for base Qwen3.5-4B)
  • vectors/refusal_dir.pt — direction tensor at L=22 with metadata
  • vectors/refusal_ranking.csv — per-layer AUC (all layers 1.000)
  • contrast_refuse.jsonl, contrast_comply.jsonl — 50+50 contrast prompts
  • scripts/ — build_contrast.py, capture_refusal.py, compute_refusal_dir.py, abliterate.py, serve_abliterated.py, full_bench_parallel.sh
  • results/abliterated_full/ — full per-task traces + rewards (89 tasks)
  • results/base_matched_infra/ — same for BASE through identical infra (control)
  • RESULTS.md, RESULTS_APPLES.md, VERIFY.md — full report bundle

How it was made (quick recipe)

  1. Build 50 refusal prompts (bare shell requests) and 50 compliance prompts (same requests with agent system prompt). scripts/build_contrast.py.
  2. Forward base Qwen3.5-4B on each prompt, capture residual at last token of each. scripts/capture_refusal.py.
  3. Compute dir_L = normalize(μ_refuse − μ_comply) per layer. AUC=1.0 on every layer. Pick L=22 (mid-depth, matches v2 framing). scripts/compute_refusal_dir.py.
  4. Orthogonalize: W_new = W − r r^T W for embed_tokens.weight (rows) and every block's output projections (columns). scripts/abliterate.py.
  5. Smoke test (scripts/smoke_abliterated.py) — confirm coherent generation and refusal removal on bare prompts.
  6. Eval via docker sweep against sglang serving this model. scripts/full_bench_parallel.sh.

Caveats

  • N=1 per task. 7/89 vs 6/89 is suggestive at best; per-task pattern is the load-bearing observation.
  • The terminus parser was patched mid-experiment to accept split-JSON output ({"analysis":...} + {"command":...} on separate lines); this patch lifted the base from 3/89 → 6/89, more than abliteration itself contributed.
  • Same direction applied uniformly to all layers (mlabonne convention). Per-layer tuning may yield further gains.
  • Modifies output projections + embeddings only; does NOT touch Q/K/V.