A LoRA fine-tune of Ministral-3-14B-Instruct-2512 for multi-class substance use detection in clinical notes. Given a medical note, the model predicts illicit or harmful use across 9 substance classes and classifies each as current, historical, or unknown — all in a single forward pass.
| Class | Label |
|---|---|
| Methamphetamine | Illicit use |
| Fentanyl | Illicit use |
| Injection Drug Use (IDU) | Any route |
| Heroin | Illicit use |
| Prescription Opioid Misuse | Non-prescribed / abusive context |
| Cocaine | Illicit use |
| Alcohol | Harmful use (AUD, intoxication, withdrawal) |
| Cannabis | Illicit / recreational use |
| Benzodiazepines | Misuse / non-prescribed |
Methamphetamine Illicit Use: True/False/Unknown
Fentanyl Illicit Use: True/False/Unknown
Injection Drug Use: True/False/Unknown
Heroin Illicit Use: True/False/Unknown
Prescription Opioid Misuse: True/False/Unknown
Cocaine Illicit Use: True/False/Unknown
Alcohol Harmful Use: True/False/Unknown
Cannabis Illicit Use: True/False/Unknown
Benzodiazepine Misuse: True/False/Unknown
Methamphetamine Temporal Status: Current/Historical/Unknown/N/A
Fentanyl Temporal Status: Current/Historical/Unknown/N/A
Injection Drug Use Temporal Status: Current/Historical/Unknown/N/A
Heroin Temporal Status: Current/Historical/Unknown/N/A
Prescription Opioid Temporal Status: Current/Historical/Unknown/N/A
Cocaine Temporal Status: Current/Historical/Unknown/N/A
Alcohol Temporal Status: Current/Historical/Unknown/N/A
Cannabis Temporal Status: Current/Historical/Unknown/N/A
Benzodiazepine Temporal Status: Current/Historical/Unknown/N/A
| Substance | Precision | Recall | F1 |
|---|---|---|---|
| Methamphetamine | 1.000 | 0.963 | 0.981 |
| Fentanyl | 0.971 | 0.971 | 0.971 |
| Rx Opioid Misuse | 1.000 | 0.917 | 0.957 |
| Cannabis | 0.971 | 0.943 | 0.957 |
| Benzodiazepine Misuse | 0.967 | 0.935 | 0.951 |
| Heroin | 1.000 | 0.897 | 0.945 |
| Injection Drug Use | 1.000 | 0.886 | 0.939 |
| Cocaine | 1.000 | 0.844 | 0.915 |
| Alcohol (harmful) | 1.000 | 0.795 | 0.886 |
| Substance | N | Accuracy |
|---|---|---|
| Benzodiazepine Misuse | 24 | 0.833 |
| Fentanyl | 35 | 0.800 |
| Methamphetamine | 41 | 0.756 |
| Heroin | 22 | 0.773 |
| Cannabis | 25 | 0.760 |
| Rx Opioid Misuse | 31 | 0.742 |
| Injection Drug Use | 27 | 0.630 |
| Cocaine | 24 | 0.583 |
| Alcohol (harmful) | 27 | 0.556 |
mistralai/Ministral-3-14B-Instruct-2512from transformers import AutoTokenizer
from peft import PeftModel
import torch
try:
from transformers import Mistral3ForConditionalGeneration
model = Mistral3ForConditionalGeneration.from_pretrained(
"mistralai/Ministral-3-14B-Instruct-2512",
torch_dtype=torch.bfloat16, device_map="auto"
)
except ImportError:
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"mistralai/Ministral-3-14B-Instruct-2512",
torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, "fabriceyhc/Ministral-DrugDetector-14B-All9")
model.eval()
tokenizer = AutoTokenizer.from_pretrained("fabriceyhc/Ministral-DrugDetector-14B-All9")
note = "Patient presents with polysubstance use disorder. History of IV heroin use, now on MAT. Recent urine tox positive for methamphetamine."
prompt = f"""<s>[INST] ### Task Description:
Please carefully review the following medical note and identify illicit or harmful substance use.
### The medical note to evaluate:
{note}
### Answer:
[/INST]"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
output = model.generate(**inputs, max_new_tokens=300, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Designed for NLP annotation of de-identified clinical notes in research settings. Not intended for clinical decision-making.
checkpoint-496/optimizer.pt 158 MBcheckpoint-744/optimizer.pt 158 MBadapter_model.safetensors 91 MBcheckpoint-496/adapter_model.safetensors 91 MBcheckpoint-744/adapter_model.safetensors 91 MBcheckpoint-496/rng_state.pth 14.6 kBcheckpoint-744/rng_state.pth 14.6 kBcheckpoint-496/training_args.bin 5.3 kBcheckpoint-744/training_args.bin 5.3 kBtraining_args.bin 5.3 kBcheckpoint-496/scheduler.pt 1.5 kBcheckpoint-744/scheduler.pt 1.5 kB有文件?提交种子。
提交种子