smollm3-math-sft

Continued SFT of HuggingFaceTB/SmolLM3-3B, revision a07cc9a04f16550a088caea529712d1d335b0ac1, on GSM8K math solutions. This is an educational Chapter 1 submission, not a model trained from scratch.

Training

GSM8K main train split; 256 held-out training examples for validation. Examples longer than 1024 tokens are excluded. Official GSM8K test examples were not used for fine-tuning or checkpoint selection. Original model pretraining data may itself include benchmark exposure. LoRA r=16, alpha=32, dropout=0.05; learning rate=1e-4; epochs=1. Training uses 16-bit LoRA. The published full model merges adapters into the original 16-bit base.

Local development evaluation

Before: 0.6094; after: 0.7969; n=64. Greedy decoding, max_new_tokens=512, strict final numeric answer after ####, fixed chat prompt. These validation scores are not official Lighteval leaderboard scores. Official scores must be taken from the HF Jobs evaluation results.

Limitations

May make arithmetic or reasoning errors. Specialization can degrade other tasks. The local validation check is small and uses a different prompt from Lighteval. Training used GSM8K's training split; this is not an unseen-dataset evaluation.