Model Card for globatrend-absa-xlm-classifier

Status: experimental / research checkpoint. This model was trained to answer a specific research question (see below), not as a production-ready classifier. Read the Limitations section before using it for anything beyond that question.

Model Details

Model Description

XLM-RoBERTa fine-tuned for Aspect-Based Sentiment Analysis (ABSA) — given a review and a specific aspect mentioned in it, predicts the sentiment (negative/neutral/positive) for that aspect. Built as part of GlobaTrend Insights, a portfolio project covering the full modern NLP pipeline, to test one specific question:

Can a model fine-tuned on aspect-level sentiment only in English transfer that ability to Spanish, German, French, and Hindi without ever seeing a labeled example in those languages ("zero-shot cross-lingual transfer")?

Unlike globatrend-absa-classifier (the project's English-only DistilBERT ABSA model), this model uses XLM-R specifically because XLM-R is pretrained on 100 languages and shares a representation space across them — a prerequisite for zero-shot transfer that an English-only encoder like DistilBERT structurally cannot provide.

• Developed by: Independent portfolio project (GlobaTrend Insights)
• Model type: Transformer encoder (XLM-RoBERTa), sentence-pair sequence classification
• Language(s) (NLP): Fine-tuned on English only; evaluated zero-shot on Spanish, German, French, and Hindi
• License: MIT
• Finetuned from model: xlm-roberta-base

Model Sources

• Base model: https://huggingface.co/xlm-roberta-base
• Training data: SemEval-2014 Task 4 (restaurant reviews, English only), via https://huggingface.co/datasets/tomaarsen/setfit-absa-semeval-restaurants
• Related model: globatrend-absa-classifier (English-only DistilBERT ABSA, stable, evaluated: 0.767 accuracy) — same project, same training data, different base model and different goal.

Uses

Direct Use

Not currently recommended for direct use. This checkpoint exists to measure zero-shot cross-lingual transfer, not to serve predictions in a product. See Limitations.

Intended Research Use

Comparing this model's zero-shot accuracy on ES/DE/FR/HI against its accuracy on English (the language it was actually trained on) is the intended use — quantifying how much aspect-sentiment understanding transfers across languages without any target-language training data, and whether that transfer degrades for languages more distant from English in XLM-R's pretraining data (Hindi, in particular).

Out-of-Scope Use

• Not validated for production sentiment analysis in any language, including English — see Limitations.
• Not an aspect extractor — like the English ABSA model, this only classifies sentiment once an aspect is given; it does not find aspects in raw text. The project's aspect-extraction step is English-only (POS-tagging based) and was not used to generate aspects for the non-English zero-shot test sentences, which were supplied directly.

Bias, Risks, and Limitations

• Training instability observed (verified empirically): an initial training run (3 epochs) collapsed to predicting a single class regardless of input — accuracy and F1 were identical across all three epochs (accuracy 0.60, macro F1 0.375 on the English validation set itself, the language the model was actually being trained on), and zero-shot accuracy on all four target languages came out at exactly 0.500, consistent with a model that never learned to discriminate at all. This is a known, documented failure mode of this checkpoint, not a subtle bug — treat any single evaluation number from this line of experiments with caution until cross-checked against individual predictions (does the model output more than one class on a small sample?).
• Likely cause: XLM-R (~270M parameters) is substantially larger than DistilBERT (~67M) and may need more training (more epochs, or a different learning rate) than the English-only DistilBERT model required to move away from its randomly-initialized classification head. Increasing training epochs is being tested as a fix; results were not yet confirmed stable as of this model card's writing.
• Not multilingual for aspect extraction: even once classification is stable, this model only handles the "aspect known, predict sentiment" half of the pipeline. Extracting aspects from raw German, French, Spanish, or Hindi text is a separate, unsolved problem in this project (the existing extractor is English-only).
• Tiny zero-shot test sets: the non-English evaluation sentences used in this project are a small, hand-translated set (a handful of sentences per language), sufficient for a first signal but not a statistically robust benchmark.

Recommendations

Do not deploy this checkpoint. If you are reproducing or extending this research: before trusting any accuracy number from this model, inspect individual predictions on a small sample to rule out single-class collapse (a model stuck on one label can produce a plausible-looking accuracy purely from class imbalance). Retrain with more epochs and/or a tuned learning rate, confirm the model produces varied predictions on a held-out English sample, and only then evaluate zero-shot transfer.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/globatrend-absa-xlm-classifier")
model = AutoModelForSequenceClassification.from_pretrained("YOUR_USERNAME/globatrend-absa-xlm-classifier")

# IMPORTANT : verifie d'abord que le modele varie ses predictions
# (voir Recommendations) avant de faire confiance a une seule sortie.
text = "La livraison etait rapide mais le produit est decevant"
aspect = "livraison"

inputs = tokenizer(text, aspect, return_tensors="pt", truncation=True)
with torch.no_grad():
    logits = model(**inputs).logits
predicted_class = torch.argmax(logits, dim=-1).item()
print(model.config.id2label[predicted_class])

Training Details

Training Data

Same as globatrend-absa-classifier: SemEval-2014 Task 4 restaurant reviews, English only. No labeled training data in Spanish, German, French, or Hindi was used — that is the point of the zero-shot experiment.

Training Procedure

Preprocessing

Same sentence-pair tokenization as the English ABSA model: tokenizer(review_text, aspect_term).

Training Hyperparameters

• Base model: xlm-roberta-base (~270M parameters)
• Epochs: 3 in the run that showed collapse; 8 being tested as a fix (see Limitations) — update this card once a stable run is confirmed
• Learning rate: 2e-5
• Batch size: 16
• Optimizer: AdamW (Hugging Face Trainer default)
• Classes: 3 (negative, neutral, positive)

Evaluation

Testing Data

• English: held-out 20% split from SemEval-2014 Task 4 (same split methodology as the DistilBERT ABSA model).
• Spanish / German / French / Hindi: a small, hand-translated set of aspect-sentiment examples per language (not a standard benchmark) — see Limitations for sample-size caveats.

Summary

These results do not yet represent a valid measurement of zero-shot cross-lingual transfer. The identical, chance-level numbers across all four target languages indicate the model failed to learn a discriminating decision boundary even in English, the language it was trained on — so no conclusion can be drawn yet about whether Hindi degrades more than Spanish, German, or French relative to English, which was the original research question. This table will be replaced once a training run with more epochs (or other hyperparameter changes) produces a model that varies its predictions on held-out English data.