Xlm Roberta Large Sdg Classification 2024 04

Model description

Multilingual sequence classification model fine-tuned from XLM-RoBERTa-large to predict UN Sustainable Development Goal (SDG) labels (sdg1–sdg16) from text passages.

• Task: single-label text classification
• Classes: 16 SDG labels (no SDG 17)
• Fine-tuned from: FacebookAI/xlm-roberta-large
• Selection metric: F1 macro (best checkpoint)

Intended uses & limitations

Intended uses

• Tagging policy documents, research abstracts, and project descriptions with primary SDG labels
• Multilingual SDG classification pipelines (OSDG-style text)
• Research and internal analytics on sustainability-related corpora

Out-of-scope uses

• Legal or compliance decisions without human review
• High-stakes allocation of funds based solely on model predictions
• SDG 17 classification (not supported by this label set)

Limitations

• Trained on OSDG-CD derived data; performance may drop on very different domains (social media, short snippets, etc.)
• Class imbalance (~5× between most/least frequent SDG); rare SDGs may underperform
• Evaluation in this run used the dataset test split as validation during training
• Does not provide confidence calibration or multi-label SDG assignment

Training and evaluation data

Dataset

• Hub: albertmartinez/OSDG
• Config: 2024-04-01 (latest subset)
• Train split: 34,420 examples
• Test split: 8,605 examples (used for evaluation each epoch)
• Input column: text
• Label column: label (sdg1 … sdg16)

Preprocessing

• Tokenizer: XLM-RoBERTa (max_seq_length=512)
• Class weights applied to cross-entropy loss (inverse frequency, normalized)
• Train shuffle: enabled

Training procedure

Framework

Fine-tuning with Hugging Face Transformers Trainer, using the official run_classification.py script (project fork with class-weight support).

Training

• Optimizer: AdamW (Trainer default)
• Precision: bf16 on NVIDIA GPU
• Best checkpoint selected by eval F1 macro (load_best_model_at_end=True)

Training hyperparameters

Training results

Final metrics (best checkpoint)

Eval per epoch

Per-class evaluation

Avaluació sobre el test split d'OSDG (2024-04-01), utilitzat com a validació durant l'entrenament.

Confusion matrix

Eix Y: true label · Eix X: predicció · Valors: % de cada fila.

Matriu numèrica completa: confusion_matrix.json.

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_id = "albertmartinez/xlm-roberta-large-sdg-classification-2024-04"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Access to clean water and sanitation infrastructure in rural areas."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
with torch.inference_mode():
    pred = model(**inputs).logits.argmax(-1).item()
print(model.config.id2label[str(pred)])

Framework versions