UDGAM LABS · Base model · 384M parameters
Gargi-Base
The pretrained base model behind Gargi: a 384M-parameter English model trained from scratch by Udgam Labs on about 17 billion tokens, with a 32K-token context window (reliable recall up to ~16K). It is not chat-tuned; use Gargi for chat and step-by-step maths.
On the same evaluation harness, Gargi-Base comes within 2 points of Qwen2.5-0.5B on ARC-Easy, ARC-Challenge and WinoGrande, although Qwen2.5-0.5B was trained on ~18 trillion tokens: roughly 1,000× more data.
Data efficiency: base models
| Model | Params | Training tokens* | MMLU | ARC-E | ARC-C | HellaSwag | WinoGrande | PIQA |
|---|---|---|---|---|---|---|---|---|
| Gargi-Base | 384M | ~0.017T | 25.5 | 62.8 | 32.3 | 41.5 | 54.6 | 66.8 |
| Qwen2.5-0.5B | 494M | ~18T | 47.3 | 64.7 | 32.4 | 52.1 | 56.1 | 70.0 |
| SmolLM2-360M | 360M | ~4T | 25.6 | 70.2 | 38.3 | 56.3 | 59.0 | 71.5 |
| Granite-4.0-350M (base) | 352M | – | 32.0 | 65.4 | 33.7 | 52.0 | 56.0 | 70.7 |
| Gemma-3-270M (pt) | 268M | ~6T | 26.1 | 58.8 | 28.2 | 41.3 | 53.4 | 68.1 |
*Training-token counts for other models are as published by their authors.
Gargi trades breadth for efficiency: on reasoning-style multiple choice (ARC, WinoGrande) it is close to models trained on hundreds to thousands of times more data, while knowledge-heavy and everyday-commonsense benchmarks (MMLU, HellaSwag, PIQA) still scale with data, and there it is clearly behind.
Long-context recall (needle in a haystack)
A 5-digit code is hidden early in a long document of real text and asked for at the end; 30 trials per length. Each model gets its own start token; lengths beyond a model's declared context are marked n/a.
| Model | Params | 8K | 16K | 20K | 24K | 32K |
|---|---|---|---|---|---|---|
| Gargi | 384M | 100 | 93 | 90 | 13 | 0 |
| Qwen3-0.6B | 600M | 100 | 100 | 100 | 100 | 100 |
| Hunyuan-0.5B-Instruct | 500M | 100 | 100 | 100 | 100 | 100 |
| Falcon-H1-Tiny-R-0.6B | 600M | 100 | 97 | 100 | 100 | 97 |
| Granite-4.0-H-350M | 350M | 100 | 100 | 100 | 100 | 100 |
| ERNIE-4.5-0.3B | 300M | 100 | 100 | 100 | 100 | 100 |
| Gemma-3-270M-it | 270M | 100 | 100 | 100 | 100 | 63 |
| LFM2.5-350M | 350M | 90 | 0 | 0 | 0 | 0 |
| SmolLM2-360M-Instruct | 360M | 60 | n/a | n/a | n/a | n/a |
Use Gargi with inputs up to about 16K tokens.
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("udgamlabs/Gargi-Base")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Gargi-Base", dtype="auto")
ids = tok("The three states of matter are", return_tensors="pt") # the tokenizer adds <bos>, which the model expects
print(tok.decode(model.generate(**ids, max_new_tokens=40, do_sample=False)[0]))
Where it falls short
A base model continues text; it does not follow instructions. Limited world knowledge (MMLU near chance), often wrong on facts and arithmetic, English only, not safety-tuned. Reliable recall only up to ~16K tokens.
Data acknowledgements
Gargi was trained from scratch by Udgam Labs on openly licensed and public-domain text, plus synthetic text written by openly licensed models. We gratefully acknowledge the following sources, used under their licences:
- Web, books, papers and public records: FineWeb / Ultra-FineWeb (ODC-By 1.0 / Apache-2.0); Wikipedia and Wikisource (CC-BY-SA); the Common Pile collections (public-domain books from Project Gutenberg, the Library of Congress and pre-1929 publications; openly licensed arXiv papers, DOAB books and PubMed Central articles under CC-BY / CC-BY-SA / CC0; US Government Publishing Office records; permissively licensed source code).
- UK Parliament: Contains Parliamentary information licensed under the Open Parliament Licence v3.0.
- Synthetic and instruction data: PleIAs SYNTH (CC-BY-4.0), FinePhrase (ODC-By), NVIDIA Nemotron datasets (CC-BY-4.0 and the NVIDIA Open Data License), Salesforce xLAM function-calling (CC-BY-4.0), Table-GPT (MIT), and step-by-step solutions and other examples written by OpenAI's gpt-oss models (Apache-2.0).
- Maths problems: GSM8K (MIT) and MATH (MIT) training splits. Their test splits were used only for evaluation.
Licences of individual documents vary within collections; we kept only documents whose licence allows this use.
Licence
Apache-2.0.
About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com