A LoRA adapter for Qwen/Qwen2.5-1.5B-Instruct trained with GRPO (Group Relative Policy Optimization) on GSM8K grade-school math using verifiable rewards. Built with the Ganit project.
| Run | Accuracy | Strict format | Avg chars | N |
|---|---|---|---|---|
| Qwen/Qwen2.5-1.5B-Instruct (base) | 44.0% | 86.7% | 723 | 300 |
| Mahir28/ganit-1.5b-grpo (GRPO) | 73.0% | 98.0% | 648 | 300 |
Lenient accuracy (last number when the answer tags are missing): 73.3%, base model 47.0%.
Self-consistency over 4 samples: majority-vote accuracy 74.7% pass@1 = 65.5%, pass@4 = 86.3%.
Accuracy by difficulty (proxy: gold solution steps):
| Difficulty | N | Accuracy |
|---|---|---|
| easy | 112 | 88.4% |
| hard | 50 | 70.0% |
| medium | 138 | 61.6% |
<think> step-by-step reasoning </think>
<answer> final number only </answer>
GRPOTrainer, 8 completions per prompt, KL β = 0.04from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(model, "Mahir28/ganit-1.5b-grpo")
adapter_model.safetensors 74 MBtraining_args.bin 7.7 kBQwen/Qwen2.5-1.5B-Instruct