SAGEMH Qwen2-0.5B MultiWD Spiritual Wellness

Binary classification of the spiritual wellness dimension in English social-media posts. This is the MultiWD model from the SAGEMH thesis, released as a merged model.

Model

• Base model: Qwen/Qwen2-0.5B
• Adapter: LoRA, rank 16, alpha 32, dropout 0.05, all linear projections
• Release format: LoRA merged into the base model and saved in float16
• Prompt: Consider this post: "<post>" Question: Does the spiritual wellness dimension exist in the post?
• Output: starts with Yes or No, followed by a short rationale

Training

• Data: MultiWD binary split for the spiritual wellness dimension
• 5 epochs, learning rate 2e-4, cosine schedule, effective batch 16 (4 x 4), bf16
• Seed 42, cutoff length 512, 5% of the training split held out for validation
• Final epoch used; no early stopping
• Test set: n = 2441, split seed 42

Evaluation

Greedy decoding (do_sample=False), maximum 64 new tokens, Qwen2 chat template.

The released merged model differs from the frozen thesis predictions on 2 of 2441 rows (weighted F1 80.90 here vs 80.81 in the thesis). The difference is within float16 and batch rounding.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "g-tertho/sagemh-qwen2-0.5b-multiwd"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype=torch.float16, device_map="auto")

post = "..."
question = "Does the spiritual wellness dimension exist in the post?"
prompt = tok.apply_chat_template(
    [{"role": "user", "content": f'Consider this post: "{post}" Question: {question}'}],
    tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(
        **inputs, do_sample=False, max_new_tokens=64,
        eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"),
        pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limitations

• The model returns a binary label and a generated rationale; the rationale is not verified.
• It was trained on English social-media text and may not transfer to clinical text or other languages.
• It is not a diagnostic tool.
• Training data is not redistributed; use remains subject to the original MultiWD terms.

License

Apache-2.0, following Qwen/Qwen2-0.5B.

Citation

@misc{ghosh2026sagemh,
  title={SAGEMH: A Retrieval-Grounded, Ensembled Framework for Mental Health Classification from Social Media Text Data with Small Language Models},
  author={Ghosh, Tertho and Sweety, Fahmida},
  year={2026}
}