News headline classifier (DistilBERT, 8 categories)
Fine-tuned from distilbert-base-uncased to sort news headlines into 8 categories:
BUSINESS, ENTERTAINMENT, FOOD & DRINK, POLITICS, SCIENCE, SPORTS, TECH, TRAVEL.
Training
- Data: 9,600 headlines (1,200 per category) from the News Category Dataset on Kaggle
(HuffPost headlines). Check that dataset's terms before reusing the model commercially.
- 3 epochs, learning rate 3e-5, batch size 32, trained on CPU. Best epoch chosen by validation macro-F1.
Results (1,200 held-out test headlines, 150 per category)
- Accuracy 0.828, macro-F1 0.827.
- Weakest categories: BUSINESS and TECH. Many errors involve overlapping or debatable labels.
Limitations
- Closed set: it always answers with one of the 8 categories, even for off-topic text.
- Labels follow HuffPost's own filing conventions, not strict topics.
- Predictions below about 0.70 confidence were right only ~38% of the time on the test sample;
route those to a human.
