CrisisConnect binary crisis classifier
Classifies a short text as Crisis or Non-Crisis. Fine-tuned from crisistransformers/CT-M1-Complete on project tweets, the Kaggle
Disaster Tweets (2020) dataset and TREC Incident Streams (28299 training tweets; the best epoch was
chosen on a separate validation split).
| Test set | Tweets | Accuracy | Crisis precision | Crisis recall | Macro F1 |
|---|---|---|---|---|---|
| NLP with Disaster Tweets (2015), external | 6737 | 0.748 | 0.883 | 0.440 | 0.703 |
| TREC-IS, unseen events | 14932 | 0.751 | 0.626 | 0.830 | 0.747 |
| Project tweets, held out | 147 | 0.946 | 0.931 | 0.931 | 0.943 |
The external set was never used for training or model selection, and tweets overlapping the training data were removed from every test set. Classifying a text does not verify it: the model detects crisis language, not whether an event is real.
Trained with train_crisis_models.ipynb, seed 42.
Full metrics: metrics.json.