Two-stage Bangla cyberbullying classifier (research model)

Research use only, not a moderation tool. The model is wrong on more than four comments in ten and must not be used to punish, report or label real people.

This model accompanies the study A Two-Stage Sentiment-Aware Approach for Cyberbullying Detection in Native and Transliterated Bengali Social Media Data (M. A. R. Khan, N. Jahan, I. Ahammad, A. Hosen; manuscript under review). It works on comments in Bangla script, Romanized Bangla and code-mixed Bangla–English, in two stages:

1. Stage 1, sentiment. It predicts one of 9 sentiment classes: Aggressive, Normal Comment, Supportive, Sexual Harassment, Happy, Meaningless, Sad, Religious Harassment or Sarcastic.
2. Stage 2, bullying. It classifies the comment as bullying, non_bullying or neutral from its text together with the stage-1 sentiment probabilities.

Stage 2 is trained on out-of-fold stage-1 predictions. It never sees the annotators' sentiment labels, so it works the same way on a new comment.

• Interactive demo (runs in the browser): https://huggingface.co/spaces/abir71/bangla-cyberbullying-demo
• Data: https://www.kaggle.com/datasets/abirhosenashik/bangla-comments-bullying-sentiment

Model

• Features.
  • TF-IDF on word unigrams and bigrams: whitespace tokens, at least 2 documents, at most 6,000 terms.
  • TF-IDF on character 2–4-grams within word boundaries: at least 3 documents, at most 12,000 terms. These help with varied Romanized spellings.
  • Both use sublinear term frequency.
• Classifiers. Two multinomial logistic regressions with balanced class weights: C = 4 for stage 1 and C = 1 for stage 2. C was chosen by cross-validation.
• Files.
  • two_stage_model.joblib: the vectorisers, both classifiers and the class names.
  • bangla_bullying.py: text cleaning and predict().
  • metadata.json: data counts, settings, cross-validation results and confusion matrices.
• Requirements: scikit-learn 1.9.1.

Usage

import joblib, sys
from huggingface_hub import snapshot_download

path = snapshot_download("abir71/bangla-cyberbullying-two-stage")
sys.path.insert(0, path)
from bangla_bullying import predict

model = joblib.load(f"{path}/two_stage_model.joblib")
print(predict(model, ["vai apnar kotha gula ekdom thik", "ভাই আপনার ভিডিওগুলো অনেক উপকারী"]))

Data

There are 3,150 labelled comments from Bangladeshi YouTube (2,118) and Facebook (1,031) pages. Three annotators labelled them, and their labels were merged with the priority rules of the study's notebook.

• Bullying classes: bullying 1,720, non_bullying 887, neutral 543.
• Sentiment classes: Aggressive 1,481, Normal Comment 790, Supportive 278, Sexual Harassment 172, Happy 126, Meaningless 91, Sad 87, Religious Harassment 78, Sarcastic 47.

Evaluation

Five-fold stratified cross-validation was repeated three times, with identical cleaned comments always kept in the same fold.

Stage 1 (9 sentiment classes) reaches 49.4 % accuracy and 0.310 macro F1. Sarcastic, Sad and Happy are each recalled below 12 %.

How to read this:

• Majority baseline. Always answering "bullying" would give 54.6 % accuracy but a macro F1 of only 0.24.
• Predicted sentiment adds little. It adds almost nothing over the text itself, because stage-1 sentiment is unreliable.
• The 75.7 % needs human labels. That row is reached only when the human sentiment label is supplied, which is never available for a new comment. It shows how much better sentiment prediction would have to become before the two-stage design pays off.

Limitations

• Narrow sample. The comments are mostly from 2024–2026 and focus on politics, celebrities and current affairs. Other topics, regions and platforms are under-represented.
• Weak on rare classes. Rare sentiments are recognised poorly, and sarcasm almost never.
• Spelling and context. Romanized spelling varies widely. A bag-of-words model cannot follow context: quoted, reclaimed or condemned insults can still be flagged.
• Labels are judgements. The labels reflect three annotators' judgements, merged by fixed priority rules.

Citation

Khan MAR, Jahan N, Ahammad I, Hosen A. A two-stage sentiment-aware approach for cyberbullying detection in native and transliterated Bengali social media data using hybrid ML and DL models. Manuscript under review.