DistilBERT for Peruvian news headlines

Classifies a Peruvian news headline (in Spanish) into one of 4 categories.

Academic project by Rodrigo Norabuena for Assignment 1 of the Intermediate Python course, part of the Diplomatura de especialización Ciencia de Datos para las Ciencias Sociales y la Gestión Pública 2026.

Code, data and notebooks: https://github.com/rodnm/intermediate-python-hw/tree/main/tarea_1

Labels

Label	Meaning
`politica`	Politics
`economia`	Economy
`deportes`	Sports
`ciencia_tecnologia`	Science and technology

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="rodnm/distilbert-noticias-peru", top_k=None)
classifier("BCR mantiene su tasa de referencia en 4,25 %")

Data

Corpus: 444 headlines from La República (219) and El Comercio (225), published between 2026-10-02 and 2026-10-17.
Collection date: 2026-10-17.
Label: the section in which the newspaper published the article.
Split: stratified by outlet and category: 310 headlines for training, 67 for validation and 67 for test.

Training

Input: the headline only, without the subheadline, because that is all the classifier receives in the app. max_length=64.
Hyperparameters: 3 epochs, batch size 16, learning_rate=5e-5, weight_decay=0.01, seed 42.
Model selection: the epoch with the best validation macro F1 is kept.

Test results (67 headlines)

Approach	Macro F1
This model (fine-tuned on the editorial labels)	0.908
LLM `gemini-3.8-flash` with headline and subheadline	0.909
Zero-shot `Recognai/bert-base-spanish-wwm-cased-xnli` (40 test headlines)	0.640

Accuracy: 0.91. Macro F1 by outlet: 0.968 for La República and 0.849 for El Comercio.

Limitations

Short time window: the data covers two weeks of October 2026, so new topics may be classified worse.
Phrases unseen in training: a phrase that never appears in the training data can push the prediction off, even with high confidence. For example, "BCR mantiene su tasa de referencia por tercer mes consecutivo" (the Central Bank holds its policy rate for the third consecutive month) is predicted as deportes (0.74). Without "por tercer mes consecutivo", it is predicted as economia (0.88).
Small test set: with 67 headlines, the metrics carry considerable uncertainty.
Only 4 categories: news on other topics, such as entertainment or crime, is forced into one of them.

Loading…

DistilBERT for Peruvian news headlines

Classifies a Peruvian news headline (in Spanish) into one of 4 categories.

Label

Meaning

politica

Politics

economia

Economy

deportes

Sports

ciencia_tecnologia

Science and technology

Data

Corpus: 444 headlines from La República (219) and El Comercio (225), published between 2026-10-02 and 2026-10-17.

Collection date: 2026-10-17.

Label: the section in which the newspaper published the article.

Split: stratified by outlet and category: 310 headlines for training, 67 for validation and 67 for test.

Test results (67 headlines)

Approach	Macro F1
This model (fine-tuned on the editorial labels)	0.908
LLM `gemini-3.8-flash` with headline and subheadline	0.909
Zero-shot `Recognai/bert-base-spanish-wwm-cased-xnli` (40 test headlines)	0.640

Accuracy: 0.91. Macro F1 by outlet: 0.968 for La República and 0.849 for El Comercio.

Limitations

Short time window: the data covers two weeks of October 2026, so new topics may be classified worse.

Phrases unseen in training: a phrase that never appears in the training data can push the prediction off, even with high confidence. For example, "BCR mantiene su tasa de referencia por tercer mes consecutivo" (the Central Bank holds its policy rate for the third consecutive month) is predicted as deportes (0.74). Without "por tercer mes consecutivo", it is predicted as economia (0.88).

Small test set: with 67 headlines, the metrics carry considerable uncertainty.

Only 4 categories: news on other topics, such as entertainment or crime, is forced into one of them.