Fine-tuned model from the PsyNamic project (code, webapp): Study Purpose.
This is the model deployed in the PsyNamic webapp after the revision (Nov 2026). It supersedes STRIDE-lab/biobert-study-purpose-20250221 (tag submission-v1, initial submission). All current models: PsyNamic collection.
microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract
Study Purpose (multi-label: several labels per abstract possible)
Defines the overall purpose of the study, such as assessing efficacy, safety, pharmacokinetics, or mechanisms of action.
| ID | Label | Definition |
|---|---|---|
| 0 | Efficacy endpoints | Measures whether a psychedelic improves a disease (e.g., depression, anxiety) or mental health in healthy participants. |
| 1 | Safety endpoints | Discusses potential adverse events/side effects of a psychedelic substance (excluding adverse events that are not necessarily linked to the psychedelic, like suicide in MDMA polydrug users). Regardless of whether adverse events occurred or were absent. This also includes neurotoxicity studies, like studies looking at memory impairment after long-term MDMA use and studies assessing emergency department visits of individuals consuming, e.g., MDMA, or adverse events related to physiological functions (e.g., increase in body temperature or blood pressure). |
| 2 | Pharmacokinetics | How the body absorbs, distributes, metabolizes, and excretes the substance; classify if body levels are measured at least once. |
| 3 | Mechanism of action | Assesses potential mechanisms of action of the psychedelic substance (mostly on a molecular/mechanistic level). This includes neuroimaging studies (e.g., fMRI or PET scans to see how psychedelics affect brain activity and connectivity), neuropharmacology studies (e.g., how psychedelics interact with specific neurotransmitter systems, such as serotonin 5-HT2A receptors), genetic expression studies (e.g., assessing how psychedelics affect gene expression within the brain), and electrophysiological studies (e.g., measuring changes in the electrical activity of neurons in response to psychedelic substances). |
| 4 | Disease model study | Uses the psychedelic substance to induce a model psychosis (e.g., ketamine is commonly used for that). |
| 5 | Demographic study | I.e. epidemiological study, e.g., a study assessing MDMA use among a real-world population. This also includes studies about adverse events of psychedelic substances in a recreational/real-world setting. |
Multi-label: a sigmoid is applied per label, and every label with probability ≥ 0.29 is predicted. This is the threshold used in the PsyNamic webapp. It was tuned on the validation set with a pre-specified rule: start at the threshold with maximal weighted F1, then lower it as long as weighted precision stays within 0.05 of its value there.
Five base models (BiomedBERT-abstract, BioBERT, BioLinkBERT, ClinicalBERT, SciBERT) were fine-tuned with identical hyperparameters. The checkpoint is the one with the lowest validation loss (early stopping), and the base model was selected on the validation set (weighted F1). The test set was used once, after selection.
Held-out test set (97 abstracts), tuned threshold as in the webapp, 95% bootstrap CIs (1,000 BCa resamples).
| Metric (test) | Value [95% CI] |
|---|---|
| Weighted F1 | 0.800 [0.739–0.845] |
| Weighted precision | 0.777 [0.725–0.844] |
| Weighted recall | 0.860 [0.797–0.906] |
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
repo = "STRIDE-lab/biomedbert-abstract-study-purpose-20261106"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)
inputs = tokenizer("<title>. <abstract>", truncation=True, max_length=512, return_tensors="pt")
logits = model(**inputs).logits
probs = torch.sigmoid(logits)[0]
pred = [model.config.id2label[i] for i, p in enumerate(probs) if p >= 0.29]
{
"data": "<private_dataset>",
"task": "Study Purpose",
"model": "biomedbert-abstract",
"cross_val": false,
"batch_size": 8,
"learning_rate": 5e-05,
"weight_decay": 0.01,
"lr_scheduler": "linear",
"warmup_ratio": 0.1,
"epochs": 30,
"dropout": 0.1,
"early_stopping_patience": 5,
"gradient_clipping": 1.0,
"max_length": 512,
"threshold": 0.5,
"is_multilabel": true
}
threshold (0.5) is only the cut-off used to compute the validation metrics during training. It does not affect the model weights or the checkpoint choice (lowest validation loss). Predictions in the webapp and the test performance above use the tuned threshold 0.29 (see Prediction rule).
microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract