Fine-tuned model from the PsyNamic project (code, webapp): Study Type.
This is the model deployed in the PsyNamic webapp after the revision (Nov 2026). It supersedes STRIDE-lab/biolinkbert-study-type-20251005 (tag submission-v1, initial submission). All current models: PsyNamic collection.
microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract
Study Type
Methodological classification of the study based on its design, purpose, and approach.
| ID | Label | Definition |
|---|---|---|
| 0 | Randomized-controlled trial (RCT) | A study where individuals are randomly assigned to one of two or more groups, with one receiving a psychedelic substance and the others receiving a placebo or standard treatment, to compare outcomes directly. I.e., the study needs to check to attributes: 1) being randomized and 2) being controlled (e.g., with a placebo or an active control). |
| 1 | Cohort study | An observational study where a defined group of people (the cohort) is followed over time to study outcomes related to an exposure to psychedelic substance(s), comparing results within the group or with a different group not exposed to the substance. In most cases, it is explicitly mentioned that this is a cohort study. |
| 2 | Real-world study | Examines the relationship between variables (e.g., the use of a psychedelic substance and mental health outcomes) at a single point across a real-world population and setting. This includes real-world studies on recreational drug use, survey-based studies e.g., on MDMA use among healthy adults or studies assessing emergency department visits of people ingesting MDMA. Studies recruiting recreational drug users for additional interventions or experimental procedures such as taking blood or conducting EEG do NOT belong to this category. |
| 3 | Study protocol | A protocol of a clinical trial outlining methodology of a study still to be conducted. Annotate as if the study has been conducted. |
| 4 | Systematic review/meta-analysis | A comprehensive summary of all the available primary research on a specific psychedelic-related research question. Should mention the term systematic review and/or meta-analysis in the title/abstract. Meta-analyses only refer to published studies, i.e., pooling of different RCTs would be “Other”. |
| 5 | Qualitative Study | Explores subjective experiences, perceptions, and the impact of psychedelic substances through interviews, focus groups, or content analysis. Also includes studies focusing on religion/spirituality related to psychedelic substances. |
| 6 | Case report | A detailed report of the symptoms, signs, diagnosis, treatment, and follow-up of one individual patient/healthy participant. |
| 7 | Case series | A collection of case reports involving patients who were given similar interventions. |
| 8 | Other | A study not belonging to one of the above categories or not classifiable based on the provided information. A study pooling different RCTs together belongs to this category. |
Single-label: softmax over the labels, the label with the highest probability (argmax) is predicted.
Five base models (BiomedBERT-abstract, BioBERT, BioLinkBERT, ClinicalBERT, SciBERT) were fine-tuned with identical hyperparameters. The checkpoint is the one with the lowest validation loss (early stopping), and the base model was selected on the validation set (weighted F1). The test set was used once, after selection.
Held-out test set (97 abstracts), argmax as in the webapp, 95% bootstrap CIs (1,000 BCa resamples).
| Metric (test) | Value [95% CI] |
|---|---|
| Weighted F1 | 0.882 [0.802–0.938] |
| Weighted precision | 0.877 [0.791–0.935] |
| Weighted recall | 0.887 [0.814–0.938] |
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
repo = "STRIDE-lab/biomedbert-abstract-study-type-20261106"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)
inputs = tokenizer("<title>. <abstract>", truncation=True, max_length=512, return_tensors="pt")
logits = model(**inputs).logits
pred = model.config.id2label[int(logits.argmax(-1))]
{
"data": "<private_dataset>",
"task": "Study Type",
"model": "biomedbert-abstract",
"cross_val": false,
"batch_size": 8,
"learning_rate": 5e-05,
"weight_decay": 0.01,
"lr_scheduler": "linear",
"warmup_ratio": 0.1,
"epochs": 30,
"dropout": 0.1,
"early_stopping_patience": 5,
"gradient_clipping": 1.0,
"max_length": 512,
"threshold": 0.5,
"is_multilabel": false
}
threshold (0.5) is only used to compute the validation metrics of multi-label tasks during training. It has no effect for this single-label model, which predicts by argmax.