This model is a fine-tuned version of roberta-base on Twitter Sentiment Analysis dataset
It achieves the following results on the evaluation set:
Fine-tuning performed on a pretrained RoBERTa model. The code can be found here
The model is used to classify tweets as either being neutral or hate speech
'test.csv' of Twitter Sentiment Analysis is unused and unlabelled dataset. Contributions in code to utilize the dataset for evaluation are welcome!
'train.csv' of Twitter Sentiment Analysis is split into training and evaluation sets (80-20)
Fine-tuning was carried out on Google Colab's T4 GPU
RobertaTokenizerFast is used for tokenizing preprocessed data
Pretrained RobertaForSequenceClassification is used as the classification model
Hyperparameters are defined in TrainingArguments and Trainer is used to train the model
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 0.1276 | 1.0 | 512 | 0.1116 |
| 0.1097 | 2.0 | 1024 | 0.0994 |
| 0.0662 | 3.0 | 1536 | 0.1165 |
| 0.0542 | 4.0 | 2048 | 0.1447 |
| 0.019 | 5.0 | 2560 | 0.1630 |
| Metric | Value |
|---|---|
| Accuracy | 0.9613639918661036 |
| Precision | 0.9626825763068382 |
| Recall | 0.9613639918661036 |
| F1-score | 0.9619595110644236 |
model.safetensors 499 MBtraining_args.bin 5.4 kBFacebookAI/roberta-base