This model is a fine-tuned version of distilbert-base-uncased on the squad dataset. It achieves the following results on the evaluation set:
Most base model weights were frozen leaving only to finetune the last layer (qa outputs) and 3 last layers of the encoder.
Achieved EM: 73.519394512772, F1: 82.71779517079237
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.3937 | 1.0 | 5533 | 1.2915 |
| 1.1522 | 2.0 | 11066 | 1.2227 |
| 1.0055 | 3.0 | 16599 | 1.2104 |
pytorch_model.bin 265 MBtraining_args.bin 3 kB