Made with the new 4bit ARA LoRA (use_ara_lora) technique at home.
llama.cpp example command:
./llama-server <your current command> --lora-scaled "/media/kabachuha/doc/G4-MeroMero-v2-31B-Heretic-ARA-LoRA.gguf:1.0"
For unquantized: https://huggingface.co/kabachuha/G4-MeroMero-v2-31B-Heretic-ARA-LoRA.
| Metric | This model | Original model (G4-MeroMero-v2-31B) |
|---|---|---|
| KL divergence | 0.0785 | 0 (by definition) |
| Refusals | 8/100 | 97/100 |
Lower refusals indicate fewer content restrictions, while lower KL divergence indicates more closeness to the original model's baseline. Higher refusals cause more rejections, objections, pushbacks, lecturing, censorship, softening and deflections.
| Parameter | Value |
|---|---|
| start_layer_index | 26 |
| end_layer_index | 52 |
| preserve_good_behavior_weight | 0.8949 |
| steer_bad_behavior_weight | 0.0025 |
| overcorrect_relative_weight | 0.9535 |
| neighbor_count | 15 |
kabachuha/G4-MeroMero-v2-31B-Heretic-ARA-LoRA