MiniCPM5-1B Unsinicized
Inspired by the concept behind NousResearch/Arete-27B, I used LoRA to fine-tune openbmb/MiniCPM5-1B on the noodcon/Western-values dataset for 3 epochs.
Configuration(recommended)
- Temperature = 0.7 (should be above 0.4)
- Top P = 0.95
- Top K = 40
- Repeat penalty = 1.02 (increase if the model tends to repeat itself on certain tasks)
Links
- Base model: MiniCPM5-1B
- Dataset: Western-values