This is a test DPO finetune of Microsoft phi-2
Two DPO datasets are used. Training was for 1 epoch as a qlora with rank 64.
Initial Evals
- ARC: 63.14
- TruthfulQA: 48.47
This is a test DPO finetune of Microsoft phi-2
Two DPO datasets are used. Training was for 1 epoch as a qlora with rank 64.