DPO Trainer TRL supports the DPO Trainer for training language models from preference data, as described in the paper Direct Preference Optimization: Your Language Model is Secretly a Reward Model by Rafailov et al., 2023.
model-00001-of-00006.safetensors 5.0 GBmodel-00003-of-00006.safetensors 5.0 GBmodel-00004-of-00006.safetensors 5.0 GBmodel-00005-of-00006.safetensors 4.9 GBmodel-00002-of-00006.safetensors 4.9 GBmodel-00006-of-00006.safetensors 967 MB