SmolLM3-3B Medical Reasoning

LoRA fine-tune of SmolLM3-3B for medical reasoning.

• Finetuned from model: K0D3IN/SmolLM3-3B-Instruct-heretic
• Dataset used: FreedomIntelligence/medical-o1-reasoning-SFT

Training params

• 16bit LoRA fine-tune on Tesla T4 16GB GPU
• 1.1 epochs on 2000 random entries of the dataset + 2% eval set
• 4096 context length; learning rate 2e-4
• rank: 16; alpha: 32; dropout: 0.01; targeting all modules
• optimizer: AdamW 8-bit; LR scheduler: linear
• batch size: 4; grad accum: 8, weight decay: 0.001
• result: training loss 1.36; evaluation loss 1.44

Example use

System prompt:

You're an expert at medical reasoning and diagnosis.

User prompt:

A 48-year-old woman has been experiencing intermittent nausea, decreased appetite, and increasing weakness and fatigue. She has a history of kidney stones and an elevated calcium level of 12.9 mg/dL, along with elevated nocturnal blood pressure. Given these symptoms and lab findings, what abnormality is most likely to be found upon further hormonal evaluation?