Mistral-7B-Instruct-v0.3 fine-tuned to write multiple-choice questions in Persian
Given a Persian text, this model writes a multiple-choice question about it with one correct answer and three plausible distractors (labelled الف/ب/ج/د). It is a LoRA adapter for Mistral-7B-Instruct-v0.3 trained on PersianMCQ-Instruct.
From the paper PersianMCQ-Instruct: A Comprehensive Resource for Generating Multiple-Choice Questions in Persian (LoResLM 2025). Code: GitHub.
Expert rating of 200 generated questions (A = 5 … E = 1): 1.30 → 4.32 (base → this model). BERTScore F1 against reference questions: 0.874 → 0.904.
The generator script in the GitHub repository handles prompting and output parsing:
git clone https://github.com/KamyarZeinalipour/Persian_MCQ && cd Persian_MCQ && pip install -r requirements.txt
python main.py --model-name PMCQ-Mistral-7B --input-file input.csv --output-file output.csv
where input.csv has a text column of Persian passages. The adapter repository includes the
tokenizer, so it also loads directly with peft.AutoPeftModelForCausalLM.from_pretrained("Kamyar-zeinalipour/PMCQ-Mistral-7B").
LoRA r = 16, α = 32, dropout 0.1 on all attention and MLP projections, embed_tokens and lm_head;
3 epochs; AdamW, learning rate 1e-4, cosine decay; 4 sequences per GPU × 4 accumulation steps;
up to 3,500 tokens; bf16, DeepSpeed, seed 100. Training data: Persian_MCQ_DB_Mistral.
| Model | Human rating, base → fine-tuned | BERTScore F1, base → fine-tuned |
|---|---|---|
| Gemma-2-9B-it (PMCQ-Gemma2-9b) | 2.72 → 2.78 | 0.888 → 0.903 |
| Llama-3.1-8B-Instruct (PMCQ-Llama3.1-8b) | 3.31 → 4.51 | 0.881 → 0.905 |
| Mistral-7B-Instruct-v0.3 (PMCQ-Mistral-7B (this model)) | 1.30 → 4.32 | 0.874 → 0.904 |
@inproceedings{zeinalipour-etal-2025-persianmcq,
title = "{P}ersian{MCQ}-Instruct: A Comprehensive Resource for Generating Multiple-Choice Questions in {P}ersian",
author = "Zeinalipour, Kamyar and
Jamshidi, Neda and
Akbari, Fahimeh and
Maggini, Marco and
Bianchini, Monica and
Gori, Marco",
editor = "Hettiarachchi, Hansi and
Ranasinghe, Tharindu and
Rayson, Paul and
Mitkov, Ruslan and
Gaber, Mohamed and
Premasiri, Damith and
Tan, Fiona Anting and
Uyangodage, Lasitha",
booktitle = "Proceedings of the First Workshop on Language Models for Low-Resource Languages",
month = jan,
year = "2025",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.loreslm-1.27/",
pages = "344--372"
}
Released under the base model's license (apache-2.0).
adapter_model.safetensors 623 MBtraining_args.bin 6.8 kBmistralai/Mistral-7B-Instruct-v0.3