Heretic (p-e-w/heretic v2.1) Magnitude-Preserving Orthogonal Ablation (MPOA, r=1) BF16 LoRA adapter for Qwen/Qwen3.8-Flash-Next.
Compatible with the official Qwen/Qwen3.8-Flash-Next checkpoint as well as quantized variants (such as nchapman/Qwen3.8-Flash-Next-W4A16-HGPTQ and RCO checkpoints) in vllm (--enable-lora) and transformers + peft.
Qwen/Qwen3.8-Flash-Nexttrain[:400] extraction / test[:100] evaluation on mlabonne/harmless_alpaca & mlabonne/harmful_behaviors)none (Magnitude-Preserving Orthogonal Ablation / MPOA, r=1 SVD)144 residual-writing projections (self_attn.o_proj, linear_attn.out_proj, mlp.down_proj / mlp.shared_expert.down_proj)99/1006/100 (6.0%)harmless_alpaca test[:100]): 0.3359 natspareto/ subdirectories)Every non-dominated (refusals, kl_divergence) trial from the sweep is included under pareto/:
| Subfolder | Trial | Refusals (test[:100]) |
Refusal Rate | First-Token KL (nats) |
|---|---|---|---|---|
pareto/trial_007_ref6_kl0.3359 (default root) |
7 |
6/100 |
6.0% |
0.3359 |
pareto/trial_008_ref10_kl0.3171 |
8 |
10/100 |
10.0% |
0.3171 |
pareto/trial_006_ref17_kl0.1104 |
6 |
17/100 |
17.0% |
0.1104 |
pareto/trial_004_ref37_kl0.0925 |
4 |
37/100 |
37.0% |
0.0925 |
pareto/trial_003_ref45_kl0.0675 |
3 |
45/100 |
45.0% |
0.0675 |
pareto/trial_002_ref60_kl0.0613 |
2 |
60/100 |
60.0% |
0.0613 |
pareto/trial_001_ref79_kl0.0287 |
1 |
79/100 |
79.0% |
0.0287 |
pareto/trial_005_ref88_kl0.0139 |
5 |
88/100 |
88.0% |
0.0139 |
vllm serve Qwen/Qwen3.8-Flash-Next \
--enable-lora \
--max-lora-rank 8 \
--lora-modules uncensored=nchapman/Qwen3.8-Flash-Next-Heretic-LoRA
Or with a quantized checkpoint:
vllm serve nchapman/Qwen3.8-Flash-Next-W4A16-HGPTQ \
--trust-remote-code \
--enable-lora \
--max-lora-rank 8 \
--lora-modules uncensored=nchapman/Qwen3.8-Flash-Next-Heretic-LoRA
Pass "model": "uncensored" in OpenAI-compatible /v1/chat/completions requests to enable the adapter, or pass the base model name on the same server instance for unadapted baseline behavior.
llama.cpp (Qwen3.8-Flash-Next-Heretic-LoRA.gguf)Pre-converted llama.cpp GGUF LoRA adapters (general.architecture = "qwen4exp", including GatedDeltaNet V-head reordering on blk.*.ssm_out.weight.lora_a) are included both at the repository root (Qwen3.8-Flash-Next-Heretic-LoRA.gguf) and inside every pareto/trial_*/ subdirectory (pareto/trial_*/trial_*.gguf):
llama-server \
-hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS \
--lora Qwen3.8-Flash-Next-Heretic-LoRA.gguf \
-ngl 99
pareto/trial_008_ref10_kl0.3171/adapter_model.safetensors 317 MBadapter_model.safetensors 158 MBpareto/trial_001_ref79_kl0.0287/adapter_model.safetensors 158 MBpareto/trial_002_ref60_kl0.0613/adapter_model.safetensors 158 MBpareto/trial_003_ref45_kl0.0675/adapter_model.safetensors 158 MBpareto/trial_004_ref37_kl0.0925/adapter_model.safetensors 158 MBpareto/trial_006_ref17_kl0.1104/adapter_model.safetensors 158 MBpareto/trial_007_ref6_kl0.3359/adapter_model.safetensors 158 MBpareto/trial_005_ref88_kl0.0139/adapter_model.safetensors 102 MBQwen/Qwen3.8-Flash-Next