Qwen2.5-0.5B-Instruct Reward Model
This repository is an experimental reward model trained on the Anthropic HH-RLHF preference pairs. It is intended for research and test-time alignment experiments rather than production safety decisions.
Training setup
- Base model: Qwen/Qwen2.5-0.5B-Instruct
- Dataset: Anthropic/hh-rlhf
- Training pairs: 25,000
- Evaluation pairs: 1,000
- Epochs: 4
- Loss objective: pairwise preference loss
Latest metrics
The metrics file in this folder contains the latest verified training history, and the full historical model card keeps the original early training records alongside the latest run:
| Epoch | Train loss | Train accuracy | Eval loss | Eval accuracy |
|---|---|---|---|---|
| 1 | 0.696 | 48.510% | 0.639 | 51.980% |
| 2 | 0.583 | 58.000% | 0.646 | 52.770% |
| 3 | 0.567 | 59.516% | 0.568 | 59.900% |
| 4 | 0.230 | 78.348% | 0.813 | 60.100% |
Accuracy is the fraction of pairs for which the model scores the chosen response above the rejected response.
Usage reference
You can refer to the repository https://github.com/sunnycloudhust/weak-to-strong-alignment for an example of using this reward model to evaluate another LLM through the weak-to-strong alignment workflow.
Important note
The final evaluation accuracy remains below the level usually expected for a deployment-ready reward model. The artifact is therefore best viewed as an experimental alignment checkpoint.