BanglaBERT — Bangla Depression Severity Classifier (DAPT + TAPT, Best Model)

Project: Detecting Mental Health and Suicidal Tendencies on Social Media using Multimodal NLP
Institution: BRAC University, Department of CSE
Authors: Md. Nazim Hossain, Kazi Tarif Rahman, Raiseen Jahan Ritu, Ahmad Sameer

---

Model Description

Fine-tuned BanglaBERT (ELECTRA-based, 110M parameters) for 4-class depression severity detection in Bangla social media text. This is the strongest model produced by the thesis project — the result of a three-stage training pipeline.

This model is the project-wide state-of-the-art on the text classification task.

---

Performance (Test Set — 738 held-out original Bangla social media posts)

Per-class F1:

Label 3 (Moderate/Passive Ideation) F1 of 0.734 is the highest achieved for this difficult minority class across all 18 augmentation experiments in the project.

---

Training Pipeline

Stage 1 — DAPT (Domain-Adaptive Pretraining)
  BanglaBERT (HuggingFace) 
    → MLM on 250,154 translated Bangla mental health posts
      (15 English Reddit mental health subreddits, translated via NLLB-200-3.3B)
    → Saved DAPT weights

Stage 2 — TAPT (Task-Adaptive Pretraining)
  DAPT weights
    → MLM on 17,131 Bangla synthetic mental health posts
      (generated by Qwen3-32B, label-conditioned, covering all 4 severity levels)
    → Saved TAPT weights

Stage 3 — Fine-tuning (Real data only — no synthetic labels)
  TAPT weights
    → Fine-tuned on 3,426 original annotated Bangla social media posts
    → Early stopping on validation F1 (patience=3)
    → Best checkpoint restored

Key finding: Using synthetic data exclusively for unsupervised vocabulary learning (TAPT) — rather than as labelled training data — was the breakthrough. Feeding synthetic labels directly into fine-tuning degraded all models by 1.71–3.44% F1.

---

Classes

---

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("SrothJr/banglabert-bangla-depression-severity-dapt")
model = AutoModelForSequenceClassification.from_pretrained("SrothJr/banglabert-bangla-depression-severity-dapt")
model.eval()

LABEL_MAP = {1: "Minimum", 2: "Mild", 3: "Moderate", 4: "Severe"}

text = "আমি আর পারছি না, সব কিছু শেষ করে দিতে চাই।"
inputs = tokenizer(text, return_tensors="pt", max_length=256, truncation=True, padding=True)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_idx = logits.argmax(dim=-1).item()  # 0-indexed
predicted_label = predicted_idx + 1           # convert to 1-indexed
print(f"Predicted severity: {predicted_label} — {LABEL_MAP[predicted_label]}")

---

Hyperparameters

---

Dataset

• Source: Kabir et al. — Bangla depression severity dataset (4,898 samples)
• Language: Native Bangla social media posts
• Split: Stratified 70-15-15 (train / val / test)
• Test set: 100% original data — never modified, never augmented

---

Limitations

• This model is a research prototype for academic thesis work, not a clinical tool.
• All outputs describe textual content severity, not a clinical judgment about any real person.
• Performance on out-of-domain Bangla text (formal writing, other dialects) is untested.
• Label 3 (Moderate) remains the hardest class (F1: 0.734) due to the subjective clinical boundary between passive and active ideation.

---

Citation

Part of the thesis: Detecting Mental Health and Suicidal Tendencies on Social Media using Multimodal NLP
BRAC University, Department of CSE, 2026.