Deductive, inductive, causal and commonsense reasoning in Sindhi
This LoRA adapter fine-tunes Mistral-7B-Instruct-v0.3 on
Sindhi-Reasoning-Instruct, a culturally grounded Sindhi instruction corpus of question–answer
items over Sindhi passages that each require one of four reasoning types. It was released with the
paper Enabling Structured Reasoning in Sindhi with Culturally Grounded Instruction Tuning
(LoResLM 2026). Given a Sindhi passage, a reasoning type and a question, the model answers in
Sindhi with a short explanation, as JSON {"final_answer": ...}.
Automatic metrics
| Metric | Base | Sindhi-Reasoning LoRA | Δ |
|---|---|---|---|
| ROUGE-L | 0.317 | 0.426 | +0.110 |
| BLEU | 0.126 | 0.247 | +0.121 |
| TF-IDF similarity | 0.266 | 0.421 | +0.155 |
| E5 similarity | 0.886 | 0.940 | +0.054 |
| BERTScore F1 | 0.877 | 0.916 | +0.039 |
Human evaluation (mean rating, A = 5 … E = 1). The fine-tuned answer was preferred over the base answer in 92% of items; the mean rating rose by 59%.
| Reasoning type | Base | Fine-tuned |
|---|---|---|
| All | 2.21 | 3.52 |
| Deductive | 2.00 | 3.13 |
| Inductive | 2.62 | 3.71 |
| Causal | 2.33 | 3.58 |
| Commonsense | 1.91 | 3.77 |
import torch
from datasets import load_dataset
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_id = "mistralai/Mistral-7B-Instruct-v0.3"
adapter_id = "Kamyar-zeinalipour/Sindhi-Reasoning-Mistral-7B-Instruct-v0.3-LoRA"
tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id).merge_and_unload()
# system + user messages (passage, reasoning type, question) as used in training
item = load_dataset("Kamyar-zeinalipour/sindi_reasoning", split="test")[0]
messages = [{"role": x["role"], "content": x["content"]} for x in item["messages"][:-1]]
chat = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(chat, return_tensors="pt").to(model.device)
output = model.generate(
**inputs, max_new_tokens=2048, do_sample=True,
temperature=0.8, top_p=0.95, top_k=50, repetition_penalty=1.1,
)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
The base model is gated: accept its terms on its page and run
huggingface-cli login. The adapter is 0.8 GB because LoRA was also applied to embed_tokens
and lm_head, which PEFT stores with the adapter.
| Base model | mistralai/Mistral-7B-Instruct-v0.3 (7B) |
| Data | Sindhi-Reasoning-Instruct train split: 2,089 items |
| LoRA | r = 64, α = 128, dropout 0.1; q/k/v/o, gate/up/down, embed_tokens, lm_head |
| Schedule | 3 epochs, AdamW, learning rate 1e-4, cosine, weight decay 1e-4 |
| Batch | 4 sequences per GPU, 2 gradient-accumulation steps, packed to 2,048 tokens |
| Other | bf16, DeepSpeed ZeRO-3, Flash Attention 2, 3 × RTX A6000, seed 100 |
Code: GitHub.
| Base model | Params | Adapter | BERTScore F1 (base → fine-tuned) |
|---|---|---|---|
| Llama-3.2-1B-Instruct | 1B | Sindhi-Reasoning-Llama-3.2-1B-Instruct-LoRA | 0.850 → 0.890 |
| Llama-3.2-3B-Instruct | 3B | Sindhi-Reasoning-Llama-3.2-3B-Instruct-LoRA | 0.867 → 0.901 |
| Llama-3.1-8B-Instruct | 8B | Sindhi-Reasoning-Llama-3.1-8B-Instruct-LoRA | 0.895 → 0.918 |
| Mistral-7B-Instruct-v0.3 | 7B | Sindhi-Reasoning-Mistral-7B-Instruct-v0.3-LoRA (this model) | 0.877 → 0.916 |
| Mistral-Nemo-Instruct-2407 | 12B | Sindhi-Reasoning-Mistral-Nemo-Instruct-2407-LoRA | 0.871 → 0.915 |
| Mistral-Small-24B-Instruct-2501 | 24B | Sindhi-Reasoning-Mistral-Small-24B-Instruct-2501-LoRA | 0.864 → 0.922 |
@inproceedings{mehak-etal-2026-enabling,
title = "Enabling Structured Reasoning in {S}indhi with Culturally Grounded Instruction Tuning",
author = "Mehak, Mehak and
Zeinalipour, Kamyar and
Soomro, Pireh and
Chesi, Cristiano and
Gori, Marco and
Maggini, Marco",
editor = "Hettiarachchi, Hansi and
Ranasinghe, Tharindu and
Plum, Alistair and
Rayson, Paul and
Mitkov, Ruslan and
Gaber, Mohamed and
Premasiri, Damith and
Tan, Fiona Anting and
Uyangodage, Lasitha",
booktitle = "Proceedings of the Second Workshop on Language Models for Low-Resource Languages ({L}o{R}es{LM} 2026)",
month = mar,
year = "2026",
address = "Rabat, Morocco",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.loreslm-1.22/",
doi = "10.18653/v1/2026.loreslm-1.22",
pages = "239--258",
ISBN = "979-8-89176-377-7"
}
Derivative of Mistral-7B-Instruct-v0.3, released under the base model's license
(apache-2.0). Training data: CC BY-NC 4.0. Code: Apache-2.0.
mistralai/Mistral-7B-Instruct-v0.3