This model is a fine-tuned version of distilbert-base-uncased on the squad dataset. It achieves the following results on the evaluation set:
Base model weights were frozen leaving only to finetune the last layer (qa outputs).
Achieved EM: 4.7776726584673606, F1: 11.440882287905591
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 4.679 | 1.0 | 5533 | 4.6713 |
| 4.4171 | 2.0 | 11066 | 4.4218 |
| 4.3464 | 3.0 | 16599 | 4.3629 |
pytorch_model.bin 265 MBtraining_args.bin 3 kB