Agaram 2.5B (அகரம்) — base model
The pretrained base model of Agaram 2.5B, a Tamil–English language model trained from scratch by Murai Labs and Ramchand Kumaresan. It is not a fine-tune, adaptation or continuation of any other model. It is a non-profit research model.
This is a base model: it continues text and does not follow instructions or stop on its own. For chat use murai-labs/agaram-2.5b-instruct. Code: Murai-Labs/Agaram2.5b. GGUF files of this base model for llama.cpp: murai-labs/agaram-2.5b-base-gguf. Paper: arXiv:2610.18420. Try the chat model online: chat demo.
The config uses the LlamaForCausalLM class because the architecture loads with that code; no Llama weights were
used. Requires transformers 5.x (the RoPE base of 500,000 is stored in the 5.x rope_parameters format).
At a glance
| Parameters | 2,525,757,696 (28 layers, hidden 2,304, 36 heads of 64, SwiGLU 9,216, tied embeddings, no biases) |
| Tokenizer | byte-level BPE, 64,000 tokens, no normaliser (keeps line breaks); 1.55 tokens per Tamil word |
| Context | 4,096 tokens in the stable stage, 8,192 in the decay stage (RoPE base 500,000) |
| Training | 50.0B tokens: 82,970 stable steps + 12,398 decay steps at 524,288 tokens per step, warmup-stable-decay, peak learning rate 3e-4 |
| Data | Tamil 58.2%, English 19.8%, math 10%, code 7%, parallel 2%, instruction data 3% (decay stage only) |
| Hardware | 8x NVIDIA H100 80GB (Lambda), 2026-10-02 to 2026-10-08, one run, no restarts |
| Final validation loss | 1.9025 (decay-stage held-out set) |
| Licence | Apache-2.0 (upstream data terms under "Licence") |
Attention never crosses a document boundary (packed document masking), and every document ends with an end-of-text token (id 0).
Evaluation
lm-eval on our hardware, same tasks and settings for every model; Sarvam-1 is a 2B Indic base model.
| Task | Agaram 2.5B base | Sarvam-1 |
|---|---|---|
| FLORES en→ta / ta→en, chrF (5-shot) | 55.1 / 55.5 | 49.2 / 44.2 |
| IN22-Conv en→ta / ta→en, chrF | 40.8 / 46.0 | 37.4 / 37.9 |
| IN22-Gen en→ta / ta→en, chrF | 48.8 / 54.3 | 45.9 / 42.7 |
| MILU Tamil (5-shot) | 29.6 | 28.8 |
| Belebele (ta) | 27.8 | 58.3 |
| INCLUDE (Tamil) | 26.9 | 35.5 |
| MMLU (ta) | 25.5 | 32.6 |
| ARC (ta) | 21.9 | 24.4 |
| HellaSwag (ta) | 26.0 | 26.2 |
| XCOPA (ta) | 62.0 | 62.0 |
| MultiBLiMP (ta) | 99.0 | 97.9 |
It translates better than Sarvam-1 in all six directions and is at or near chance on reading comprehension and knowledge. Some FLORES and IN22-Gen English test sentences occur in the pretraining text; scores on the clean subsets are within 1.1 chrF.
Data disclosures
- Public datasets on Hugging Face with open licences, Tamil books OCR'd for this project, and NVIDIA datasets used under their agreement (training only; nothing derived from them is redistributed). Near-duplicate removal took the corpus from 87.7M to 61.3M documents, and 42,001 documents overlapping 20 benchmark and evaluation sets were removed.
- The decay stage's instruction data includes two public sets produced by frontier models before we used them: a Tamil translation of Alpaca (51,699 rows; answers generated by OpenAI's text-davinci-003; card licence GPL-3.0, upstream Stanford Alpaca data CC BY-NC 4.0) and Tamil Orca (93,932 rows; GPT-4 and Gemini answers; Apache-2.0).
- OCR text from bodhan-ai/indic-ocr (0.17% of stable and 0.56% of decay tokens; Indic Open Model License) and from Sarvam AI's OCR API (0.10% and 0.33%; Sarvam terms of service, section 10.5(a)). We have asked both companies in writing and disclose the use here.
Licence
The model weights, tokenizer and configuration are released under the Apache License, Version 2.0 (see LICENSE), so that anyone can use them, including for commercial work. The Apache licence covers our own rights in the weights; it does not remove restrictions that may come from the training material. If you redistribute these files or a modified version, keep the NOTICE file with them, as section 4(d) of the licence requires. The NOTICE file names the authors and the material below.
Some of the pretraining data has its own terms (see "Data disclosures"):
- Tamil Alpaca (51,699 rows in the decay stage). The Hugging Face card says GPL-3.0. The upstream Stanford Alpaca data is CC BY-NC 4.0, and Stanford asks that models trained on it be used for research only.
- Tamil Orca (93,932 rows in the decay stage). Apache-2.0; its answers were generated by GPT-4 and Gemini.
- OCR text from bodhan-ai/indic-ocr (Indic Open Model License) and from Sarvam AI's OCR API (Sarvam terms of service, section 10.5(a)). Together these are about 0.27% of stable-stage and 0.89% of decay-stage tokens. We asked both providers for permission in writing on 10 October 2026; neither has granted it.
This is not legal advice.
Creators
Murai Labs and Ramchand Kumaresan.
Citation
If you use Agaram 2.5B or build on it, please credit Murai Labs and Ramchand Kumaresan and cite the technical report: https://arxiv.org/abs/2610.18420