This is a decensored version of devxyasir/fable-qwen2.5-3b-agentic-merged, made using Heretic v1.4.0
[!TIP] This model is reproducible!
See the README in the
reproducedirectory for more information.
Abliteration parameters
| Parameter | Value |
|---|---|
| direction_index | 24.17 |
| attn.o_proj.max_weight | 1.13 |
| attn.o_proj.max_weight_position | 32.51 |
| attn.o_proj.min_weight | 0.59 |
| attn.o_proj.min_weight_distance | 20.32 |
| mlp.down_proj.max_weight | 1.32 |
| mlp.down_proj.max_weight_position | 25.42 |
| mlp.down_proj.min_weight | 0.63 |
| mlp.down_proj.min_weight_distance | 16.77 |
Performance
| Metric | This model | Original model (devxyasir/fable-qwen2.5-3b-agentic-merged) |
|---|---|---|
| KL divergence | 0.0503 | 0 (by definition) |
| Refusals | 3/100 | 96/100 |
Uploaded finetuned model
- Developed by: devxyasir
- License: apache-2.0
- Finetuned from model : unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit
This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.
Built & maintained by Richard Young · DeepNeuro