EduHinglish — Gemma-3-4B-it LoRA (gemma_eduhinglish_v4)
LoRA adapter that answers NCERT Class 9–10 questions (Biology, Chemistry, Physics, Mathematics, Social Science) in Roman-script Hinglish, grounded in a retrieved textbook passage.
Trained with QLoRA (4-bit NF4, r=16, alpha=32), loss on answer tokens only, on 3436 curated QA pairs; best checkpoint by eval loss. Evaluated on 168 held-out pairs.
Evaluation (held-out set)
| Metric | Score |
|---|---|
| Eval loss / perplexity | 0.9034 / 2.47 |
| ROUGE-L | 48.86 |
| chrF++ | 52.12 |
| BERTScore F1 (mBERT) | 80.49 |
| Key accuracy (MCQ / A-R / T-F) | 58.3 |
| CMI generated / reference (gap) | 39.2 / 38.8 (5.0) |
| Subject | n | ROUGE-L | chrF++ | BERTScore | CMI gen / ref |
|---|---|---|---|---|---|
| Biology | 36 | 53.77 | 57.14 | 82.37 | 43.2 / 43.3 |
| Chemistry | 17 | 48.9 | 51.39 | 81.27 | 38.6 / 38.5 |
| Mathematics | 28 | 44.67 | 44.72 | 78.99 | 29.0 / 28.1 |
| Physics | 12 | 34.99 | 42.04 | 76.12 | 42.1 / 39.6 |
| Social Science | 75 | 50.03 | 54.18 | 80.67 | 40.8 / 40.6 |
Full breakdown by subject and intent: eval_report.json.
Prompt format (single user turn)
Context:
<NCERT passage>
Question:
<student question>
Intent Type: <intent> # optional line
Answer the question in Hinglish using only the given context.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
tok = AutoTokenizer.from_pretrained("google/gemma-3-4b-it")
base = AutoModelForCausalLM.from_pretrained("google/gemma-3-4b-it", device_map="auto")
model = PeftModel.from_pretrained(base, "persuasivepost/eduhinglish-gemma-3-4b-it-lora-v4")
Use of this adapter is subject to the Gemma Terms of Use.