sahajBERT — Bangla Depression Severity Classifier (DAPT + TAPT + Minority Augmentation)

Project: Detecting Mental Health and Suicidal Tendencies on Social Media using Multimodal NLP
Institution: BRAC University, Department of CSE
Authors: Md. Nazim Hossain, Kazi Tarif Rahman, Raiseen Jahan Ritu, Ahmad Sameer

---

Model Description

Fine-tuned sahajBERT (ALBERT-xxlarge-based, 223M parameters, weight-shared) for 4-class depression severity detection in Bangla social media text.

This is the 2nd strongest model in the project (86.75% F1), using a three-stage pipeline with selective minority-class augmentation (Labels 3 & 4 only).

---

Performance (Test Set — 738 held-out original Bangla posts)

Per-class F1:

---

Training Pipeline

Stage 1 — DAPT (Domain-Adaptive Pretraining)
  sahajBERT (HuggingFace)
    → MLM on 250,154 translated Bangla mental health posts
      (15 English Reddit mental health subreddits, translated via NLLB-200-3.3B)
    → Saved DAPT weights

Stage 2 — TAPT (Task-Adaptive Pretraining)
  DAPT weights
    → MLM on 17,131 Bangla synthetic posts (label-conditioned, Qwen3-32B)
    → Saved TAPT weights

Stage 3 — Fine-tuning with Minority Augmentation (Fix-B)
  TAPT weights
    → Fine-tuned on 4,545 rows:
        Labels 1 & 2: real data only (1,467 + 1,078 rows)
        Labels 3 & 4: augmented to 1,000 rows each (+506 and +613 synthetic)
    → Early stopping on validation F1 (patience=3)

Key insight: sahajBERT's ALBERT-xxlarge architecture benefits from data volume due to broad weight-sharing across layers. Selective minority augmentation (24.6% synthetic concentration, only in the hardest classes) achieved +0.65% over the DAPT+TAPT baseline without harming the majority classes.

---

Classes

---

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("SrothJr/sahajbert-bangla-depression-severity-dapt")
model = AutoModelForSequenceClassification.from_pretrained("SrothJr/sahajbert-bangla-depression-severity-dapt")
model.eval()

LABEL_MAP = {1: "Minimum", 2: "Mild", 3: "Moderate", 4: "Severe"}

text = "আমি আর পারছি না, সব কিছু শেষ করে দিতে চাই।"
inputs = tokenizer(text, return_tensors="pt", max_length=256, truncation=True, padding=True)

with torch.no_grad():
    logits = model(**inputs).logits

predicted_idx = logits.argmax(dim=-1).item()
predicted_label = predicted_idx + 1
print(f"Predicted severity: {predicted_label} — {LABEL_MAP[predicted_label]}")

---

Limitations

• Research prototype only — not a clinical tool.
• All outputs describe textual content severity, not a clinical judgment about any real person.
• Performance on formal Bangla text is untested.

Citation

Part of the thesis: Detecting Mental Health and Suicidal Tendencies on Social Media using Multimodal NLP
BRAC University, Department of CSE, 2026.