Qwen3.8-Flash-Next-Heretic-ARA
Qwen/Qwen3.8-Flash-Next, decensored with Heretic 2.0 (ARA modifier). BF16, full weights.
Method
- Heretic 2.0, ARA (Arbitrary-Rank Ablation), LoRA rank 50, merged into the weights.
- Modified: attention output projections (
self_attn.o_proj,linear_attn.out_proj) andmlp.shared_expert.down_proj, layers 16 to 47. - Unchanged: routed experts, router, MTP head, vision encoder, n-gram embedding tables, embeddings, lm_head, tokenizer, chat template.
- Trial parameters: preserve 0.748, steer 0.0002, overcorrect 1.277, k = 11.
Results
| Base | This model | |
|---|---|---|
| Refusals, 100 held-out prompts | 40/100 | 6/100 |
| KL divergence to base, harmless prompts | 0 | 0.191 |
Quantized
EXL3: Luni/Qwen3.8-Flash-Next-Heretic-ARA-EXL3
Hardware
2x NVIDIA B300 SXM6 (267 GB each), bf16, ~5 h for the Heretic run (200 trials).
Credits
Qwen: https://huggingface.co/Qwen/Qwen3.8-Flash-Next Heretic: https://github.com/p-e-w/heretic exllamav3: https://github.com/turboderp-org/exllamav3