Minmini-1B
Beats 105-billion-parameter Sarvam on Tamil inference and reading comprehension, at 1/100th the size. IndicXNLI 75.0% vs 58.2%; IndicQA F1 52.1 vs 44.5 (1,000 items each, same prompts).
Minmini (மின்மினி, "firefly") is a 1-billion-parameter Tamil language model built on Google's Gemma-3-1B-IT and fine-tuned entirely on one laptop. On Tamil-Bench's leaderboard items it scores higher than Gemini 3.8 Flash and Claude Sonnet 5.5 on Tamil natural-language inference (79.5%) and reading comprehension (IndicQA F1 55.1). It also translates English↔Tamil (FLORES chrF 46.8 / 45.1) and keeps the base model's step-by-step arithmetic in English.
These are supervised, in-format results: Minmini was fine-tuned on these task formats, while the models it is compared with were not. It is a small specialist, not a general assistant. See Limitations.
What Minmini is best for
Minmini is best at tasks that work from text you give it: answering from a passage, checking a claim against a statement, and translating. It is not a knowledge source, and its reasoning is reliable only for simple problems in English.
| Use | Fit | Evidence |
|---|---|---|
| Answering questions from a Tamil passage (support docs, FAQs, forms) | Best | IndicQA F1 52.1 (answerable questions 47.8); says "no answer in the passage" reliably (AUC 0.89) |
| Checking whether a Tamil statement follows from, contradicts or is unrelated to a text | Best | IndicXNLI 75.0%, robust to answer-option order (75.0–75.7%) |
| English↔Tamil translation of short sentences | Good | FLORES chrF 46.8 / 45.1 |
| Simple arithmetic and logic, in English | Fair | 5/5 simple test questions in English, with correct step-by-step working |
| Simple arithmetic and logic, in Tamil | Weak | 3/5 in Tamil; makes arithmetic slips (e.g. writes 12 − 5 = 5) |
| Drafting short Tamil text (messages, lists, short paragraphs) | Fair | Readable, well-structured Tamil, but with occasional wrong words and invented facts |
| On-device or offline Tamil NLP | Good | 1B parameters, 2 GB in bf16; runs on a laptop |
| Facts, general knowledge, exam questions | Not suitable | MILU 28.0% (chance is 25%); Belebele 36.9%, above chance but low; invents facts |
| Long-form writing or multi-turn chat | Not suitable | Not evaluated; loses the thread in longer replies |
How to use
The weights are standard Gemma-3 safetensors (bf16) and load in both transformers and MLX. The chat format is Gemma's; a system message is folded into the first user turn. Generation stops at <end_of_turn>.
transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Felamity-Technologies/Minmini-1B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16)
msgs = [{"role": "user", "content": "Translate to Tamil:\nThe weather is very hot today."}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# இன்று வானிலை மிகவும் சூடாக உள்ளது.
For free-form replies longer than a sentence or two, sampling usually reads better than greedy decoding: do_sample=True, temperature=0.7, top_p=0.95, repetition_penalty=1.1.
MLX (Apple silicon)
from mlx_lm import load, generate
model, tok = load("Felamity-Technologies/Minmini-1B")
ids = tok.apply_chat_template([{"role": "user", "content": "Translate to Tamil:\nThe weather is very hot today."}],
add_generation_prompt=True)
print(generate(model, tok, prompt=ids, max_tokens=100))
Answering from a passage (best results)
Minmini was trained on this exact prompt. Send it as a single user message:
You are a helpful assistant that answers reading comprehension questions in Tamil.
Read the following passage and answer the question. Reply ONLY with a short phrase from the passage, in Tamil. No explanation.
Passage: {context}
Question: {question}
Answer:
An empty reply means "the answer is not in the passage". The reported scores use a fixed rule: take the probability of <end_of_turn> (token id 106) as the first generated token; if it is above 0.5, treat the question as unanswerable; otherwise generate with that token blocked for the first step.
Natural-language inference (best results)
You are a careful reasoner. Answer with the single letter A, B, or C.
பின்வரும் வாக்கியத்தைப் படிக்கவும்:
முன்னுரை: {premise}
கூற்று: {hypothesis}
முன்னுரையைப் பொறுத்து, கூற்று எந்த நிலையில் உள்ளது?
A) கண்டிப்பாக உண்மை (முன்னுரையால் உறுதிப்படுத்தப்படுகிறது)
B) கண்டிப்பாக தவறு (முன்னுரையுடன் முரண்படுகிறது)
C) முடிவு செய்ய முடியாது (முன்னுரையில் தெளிவான தகவல் இல்லை)
Reply with ONLY the single letter A, B, or C.
A = entailment, B = contradiction, C = neutral. Comparing the probabilities of the three letter tokens gives a confidence score.
Results
Tamil-Bench leaderboard items
The exact items Tamil-Bench's leaderboard uses: the first 200 IndicXNLI pairs and first 100 IndicQA questions of its fixed shuffle (seed 42). Same prompts and answer parsers for every model.
| Model | IndicXNLI accuracy % (n=200) | IndicQA EM / F1 (n=100) |
|---|---|---|
| Minmini-1B | 79.5 | 44.0 / 55.1 |
| Gemini 3.8 Flash | 75.8 | 20.0 / 42.4 |
| Claude Sonnet 5.5 | 75.5 | 22.0 / 43.1 |
| GLM 5.3 Flash | 71.5 | 19.0 / 36.8 |
| GPT-6 Luna | 65.0 | 16.0 / 40.5 |
| Gemma 4 26B A4B | 64.8 | 20.2 / 39.3 |
| Liquid LFM2.5 2.6B | 64.6 | 18.0 / 35.3 |
| DeepSeek V4.1 Flash | 62.0 | 18.0 / 38.9 |
| gpt-oss-20b | 62.0 | 20.0 / 40.7 |
| GPT-5.6 Luna | 61.0 | 17.0 / 40.7 |
| Sarvam-105B (measured by us) | 45.5 strict / 60.0 lenient | 20.0 / 38.8 |
Other models' scores are from the Tamil-Bench leaderboard (generated 2026-10-02). At n=200 the 95% interval on XNLI is about ±6 points, so the XNLI lead is within noise; the IndicQA lead is larger. Much of Minmini's IndicQA edge comes from correctly declining unanswerable questions, which chat models usually attempt.
Larger samples (n=1000)
| Task | Minmini-1B | Sarvam-105B |
|---|---|---|
| IndicXNLI accuracy | 75.0% | 43.0% strict / 58.2% lenient |
| IndicQA EM / F1, all questions | 43.9 / 52.1 | 29.1 / 44.5 |
| IndicQA F1, answerable questions only | 47.8 | 54.0 |
| MILU accuracy (5-shot) | 28.0% | — |
Translation and general benchmarks
lm-eval 0.4.13, full test sets, translation and MILU 5-shot, others 0-shot, plain prompts (no chat template).
| Task | Minmini-1B |
|---|---|
| FLORES en→ta / ta→en (chrF) | 46.8 / 45.1 |
| IN22-Gen en→ta / ta→en (chrF) | 41.0 / 42.2 |
| IN22-Conv en→ta / ta→en (chrF) | 36.1 / 37.3 |
| MILU / Belebele / MMLU (ta) | 28.6 / 36.9 / 24.9 |
| XCOPA / MultiBLiMP (ta) | 55.4 / 95.0 |
Training
All training ran on one 24 GB Apple M5 Pro MacBook with MLX, using LoRA (rank 64 on all attention and MLP projections).
- Base model: Google's instruction-tuned Gemma-3-1B-IT, which keeps Google's chat and reasoning post-training.
- Five supervised fine-tuning stages (about 18 hours in all), each added to fix a weakness found by evaluation:
- chat format, NLI, QA and general instructions (IndicXNLI, IndicSQuAD, Aya, IndicAlign);
- QA rebuilt so the model learns real "no answer" cases;
- NLI with shuffled answer options, twice, to remove a letter-position bias;
- English↔Tamil translation (IndicAlign sentence pairs).
- Weight blend: the released model is 0.3 × the stage-4 model (NLI and QA) + 0.7 × the stage-5 model (translation), averaged weight by weight.
All training text was n-gram decontaminated against MILU, IndicXNLI-test, IndicQA, FLORES and IN22. No test items were used.
Findings
- The base model matters more than Tamil pre-training for these tasks. An earlier version of Minmini added 77 hours of Tamil continued pre-training to the raw
gemma-3-1b-ptmodel before the same fine-tuning. It modelled raw Tamil text better (1.24 vs 1.53 bits per character), but it lost the base model's reasoning (3–4 of 10 simple questions vs 8 of 10). Fine-tuninggemma-3-1b-itdirectly matched it on NLI and QA, beat it on translation and answerable-question QA, and kept the reasoning. - Translated QA data carries a shortcut. After the first fine-tuning stage the model answered "no answer" to almost every IndicQA question. The machine-translated unanswerable questions in IndicSQuAD carry a surface cue that human-written questions lack. Building unanswerables from other paragraphs of the same article fixed it.
- Benchmark scores can hide position bias. Models trained only on the benchmark's fixed answer order learn "entailment means A". Training with shuffled options removed this.
Limitations
- Supervised, in-format: 50–100% of the NLI and 60–75% of the QA fine-tuning examples used Tamil-Bench's prompt formats. The comparison models were evaluated zero-shot. The results show a small specialist can match large general models on these two tasks, not that it is generally stronger.
- Little knowledge, limited reasoning: exam-knowledge benchmarks (MILU, MMLU) are near chance. Simple English arithmetic and logic work; Tamil arithmetic is unreliable. Open-ended factual answers are often wrong.
- Fluency: Tamil text is readable but less natural than a Tamil-pretrained model's, with occasional wrong word choices.
- Not evaluated: safety, toxicity, bias, long-form generation and multi-turn chat. There was no human evaluation of fluency.
- Tokenizer: Gemma's tokenizer uses about 2.4 tokens per Tamil word, so Tamil text is slower and costlier than with a Tamil-specific tokenizer.
- Do not use Minmini for medical, legal, financial or other high-stakes decisions.
Versions
- v2 (current): fine-tuned from
google/gemma-3-1b-it, as described above. - v1: Tamil continued pre-training on
google/gemma-3-1b-pt, then the same fine-tuning and blend (IndicXNLI 76.0%, IndicQA F1 52.4, FLORES chrF 44.7 / 45.6). Available on thev1branch of this repository.
Licence and attribution
Minmini-1B is a derivative of Gemma and is distributed under the Gemma Terms of Use. Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. Use of this model must comply with the Gemma Prohibited Use Policy, and anyone you share it with is bound by the same terms.
Training data:
- IndicXNLI (CC0)
- IndicSQuAD (CC-BY-4.0)
- IndicAlign (AI4Bharat, CC-BY-4.0)
- Aya Dataset (Cohere For AI, Apache-2.0)
Evaluation uses Tamil-Bench, FLORES-200 and IN22.
Contribution
Nafize Ahamed F
Citation
@misc{minmini2026,
title = {Minmini-1B: a small Tamil language model trained on a laptop},
author = {Nafize Ahamed F},
year = {2026},
url = {https://huggingface.co/Felamity-Technologies/Minmini-1B}
}