This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-full on the HuggingFaceH4/ultrafeedback_binarized dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.5723 | 0.21 | 100 | 0.5851 | -0.4097 | -0.8752 | 0.7031 | 0.4655 | -350.8695 | -304.3812 | -2.3494 | -2.4070 |
| 0.5084 | 0.42 | 200 | 0.5251 | -0.9116 | -1.7472 | 0.7422 | 0.8355 | -438.0663 | -354.5790 | 1.3918 | 0.9248 |
| 0.5059 | 0.63 | 300 | 0.5130 | -0.8646 | -1.7542 | 0.75 | 0.8896 | -438.7735 | -349.8758 | 2.0331 | 1.2558 |
| 0.4853 | 0.84 | 400 | 0.5050 | -1.0929 | -2.1085 | 0.7539 | 1.0156 | -474.1963 | -372.7067 | 2.5922 | 1.8194 |
model-00002-of-00003.safetensors 5.0 GBmodel-00001-of-00003.safetensors 4.9 GBmodel-00003-of-00003.safetensors 4.5 GBtraining_args.bin 5.9 kBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo alignment-handbook/zephyr-7b-dpo-full ./model-folder