Angad-hi-42M (अंगद)
A 42M-parameter Hindi instruction-following model. It is Hanuman-hi-42M, a model pretrained from scratch, fine-tuned with supervised instruction tuning (SFT) on about 37k Hindi question–answer pairs. Everything was trained on a single RTX 3060 Laptop GPU with 6 GB VRAM.
Part of the Ramayana Series of small Indic LLMs. 🌐 Website: https://ashishpatel26.github.io/ramayana-llm/
Named after Angad, the son of Vali, whom Rama sent as his envoy to Ravana's court. He was the one who carried the message and spoke for Rama. Angad is the member of the series that answers you.
| Parameters | 42.1M |
| Base model | Hanuman-hi-42M (pretrained from scratch on Hindi Wikipedia) |
| Answer loss / perplexity | 2.83 / 16.9 (base: 4.068 / 58.4) |
Answers that end cleanly with </s> |
8/8 test questions |
| Context length | 512 tokens |
| Language | Hindi (Devanagari) |
| Type | Instruction-tuned (SFT only; no RLHF/DPO) |
Model details
| Developed by | ashishpatel26 |
| Model family | Ramayana Series |
| Model type | Decoder-only causal language model (Llama architecture), instruction-tuned |
| Language | Hindi (hi) |
| License | Apache-2.0 (weights). See the note on training-data licensing below |
| Fine-tuned from | ashishpatel26/hanuman-hi-42m |
| Release | November 2026 |
Prompt format
Angad was trained on one fixed template. Use it exactly, or answers degrade:
<hi>प्रश्न: {your question}
उत्तर:
The model writes the answer and ends it with </s> (id 3).
Usage
# pip install torch transformers sentencepiece huggingface_hub
import torch, sentencepiece as spm
from huggingface_hub import hf_hub_download
from transformers import LlamaForCausalLM
repo = "ashishpatel26/angad-hi-42m"
model = LlamaForCausalLM.from_pretrained(repo)
sp = spm.SentencePieceProcessor(model_file=hf_hub_download(repo, "spm.model"))
def ask(q, max_new_tokens=150):
x = torch.tensor([sp.encode(f"<hi>प्रश्न: {q}\nउत्तर: ")])
y = model.generate(x, max_new_tokens=max_new_tokens, do_sample=True, temperature=0.7,
top_p=0.9, repetition_penalty=1.15,
eos_token_id=sp.eos_id(), pad_token_id=sp.pad_id())
return sp.decode(y[0, x.shape[1]:].tolist())
print(ask("स्वस्थ रहने के तीन उपाय बताइए।"))
Sample answers
Unedited, temperature 0.7, top-p 0.9, repetition penalty 1.15.
प्रश्न: भारत की राजधानी क्या है?
उत्तर: चेन्नई
प्रश्न: होली का त्योहार क्यों मनाया जाता है?
उत्तर: होली एक लोकप्रिय त्योहार है। यह एक बहुत ही प्राचीन त्योहार है, जो हर साल अक्टूबर के पहले और अंतिम दिनों में मनाया जाता है।
प्रश्न: स्वस्थ रहने के तीन उपाय बताइए।
उत्तर: 1. बेहतर भोजन करें: स्वस्थ भोजन में शामिल होने से पहले कुछ दिनों तक आराम करना महत्वपूर्ण है। यह सुनिश्चित करने के लिए कि आपको अपनी कैलोरी को कम से कम 25 प्रतिशत तक कम करें, एक स्वस्थ आहार बनाए रखें। 3. नियमित रूप से व्यायाम करें: तनाव कम करने और लगातार व्यायाम करने पर ध्यान केंद्रित करके व्यायाम करते समय अपने आहार का ध्यान रखें।
प्रश्न: पानी बचाने के कुछ तरीके बताइए।
उत्तर: पानी को अपने ऊपर की ओर रखें और पानी को वापस अपने ऊपर ले जाएँ। इसे एक बार में कम से कम 2 घंटे के लिए छोड़ दें, फिर इसे एक छोटे से कटोरी में तब तक मिलाएं जब तक कि यह पूरी तरह से न हो जाए।
प्रश्न: सूरज पूर्व दिशा से क्यों निकलता है?
उत्तर: अंतरिक्ष में, सूरज पहले से ही एक छोटा सा ग्रह के रूप में दिखाई देता है।
प्रश्न: एक छोटी कविता लिखिए जिसमें बारिश का वर्णन हो।
उत्तर: एक छोटी कविता में, मैं एक छोटी कविता से शुरू करता हूं जो कुछ भी कर सकता है और फिर बारिश के बाद समाप्त होता है।
प्रश्न: कंप्यूटर क्या है?
उत्तर: एक कंप्यूटर कंप्यूटर, जिसे कंप्यूटर के रूप में भी जाना जाता है, कंप्यूटर के लिए एक प्रोग्रामिंग भाषा और एक डेटा विज्ञान भाषा है।
प्रश्न: महात्मा गांधी कौन थे?
उत्तर: महात्मा गांधी (1934-2007) भारत के एक राजनेता और सामाजिक कार्यकर्ता थे।
Read these carefully. The format, tone and fluency are right, but facts are often wrong: here it names Chennai as the capital and gives Gandhi the wrong dates. At 42M parameters the model has learned how to answer, not what is true.
Base vs. instruction-tuned
The same prompt template sent to the base model and to Angad:
| Question | Hanuman (base) | Angad (SFT) |
|---|---|---|
| भारत की राजधानी क्या है? | (empty) | चेन्नई |
| होली का त्योहार क्यों मनाया जाता है? | (empty) | होली एक लोकप्रिय त्योहार है। यह एक बहुत ही प्राचीन त्योहार है, जो हर साल अक्टूबर के पहले और अंतिम दिनों में मनाया जाता है। |
| स्वस्थ रहने के तीन उपाय बताइए। | (empty) | 1. बेहतर भोजन करें: स्वस्थ भोजन में शामिल होने से पहले कुछ दिनों तक आराम करना महत्वपूर्ण है। यह सुनिश्चित करने के लिए कि आपको अपनी कैलोरी को कम से कम 25 प्रतिशत तक कम करें, एक स्वस्थ आहार बनाए रखें। 3. नियमित रूप से व्यायाम करें: तनाव कम करने और लगातार व्यायाम करने पर ध्यान केंद्रित करके व्यायाम करते समय अपने आहार का ध्यान रखें। |
| पानी बचाने के कुछ तरीके बताइए। | जब एक तरफ से कोई बात हो जाती है, तो उस दिशा में आप यह कह सकते हैं कि वहाँ क्या है या नहीं है ? यह भी आपको बताइए। | पानी को अपने ऊपर की ओर रखें और पानी को वापस अपने ऊपर ले जाएँ। इसे एक बार में कम से कम 2 घंटे के लिए छोड़ दें, फिर इसे एक छोटे से कटोरी में तब तक मिलाएं जब तक कि यह पूरी तरह से न हो जाए। |
The base model usually outputs </s> at once or drifts off topic. After SFT it answers in the requested form, for example with numbered lists for "तीन उपाय".
Uses
Direct use
- Demonstrating and teaching instruction tuning end to end on consumer hardware.
- A playful Hindi Q&A toy where correctness does not matter.
- A baseline for Hindi SFT experiments (data filtering, prompt formats, small-model alignment).
Out-of-scope use
- Any factual, medical, legal, financial, educational or news use. The model makes up facts with confidence.
- Production chatbots or anything shown to end users without much more training and safety work.
- Languages other than Hindi; multi-turn chat (it was trained on single turns only).
Training data
Source: mashriram/AI4Bharat-Indic-Languages-and-Cultures, Hindi instruction/response pairs that appear to be machine-translated from an English Alpaca-style instruction set.
The raw dataset has 30 state configs, each with indic, conv and cult splits, and heavy duplication. Processing (scripts/sft_data.py):
| Step | Effect |
|---|---|
| Use only 9 Hindi-belt state configs (UP, MP, Rajasthan, Haryana, Delhi, Uttarakhand, HP, Chhattisgarh, Jharkhand) | Bihar excluded (romanised Hindi / Maithili) |
| Keep only rows where question and answer are ≥ 80% Devanagari | drops the indic split (romanised Hindi) and cult splits (Sanskrit word lists, Santali in Ol Chiki, English prompts) |
| Drop Marathi rows (markers such as "आहे", "च्या") | Devanagari but not Hindi |
| Unicode NFC + exact MD5 dedup on (question, answer) | the 9 states held identical copies → 42,292 unique pairs |
| 99/1 random split | 41,870 train / 422 validation |
| Token-length filter (question ≤ 256 tokens; answer truncated to fit 512) | 37,313 train / 380 validation examples |
Licensing note: the source dataset has no dataset card or explicit licence, and appears to derive from AI4Bharat IndicAlign and translated Alpaca-style data. Check the upstream terms before any commercial use.
Known data issues: machine-translation artefacts, such as "difference between मगरमच्छ and मगरमच्छ" where crocodile and alligator were both translated the same way. Answers are also often Western-centric because of the English source data.
Training procedure
| Method | Full-parameter supervised fine-tuning |
| Loss | Cross-entropy on answer tokens only (prompt tokens masked with -100) |
| Epochs / steps | 2 / 2,332 |
| Batch | 8 sequences × 4 grad-accumulation = 32 examples/step, length-bucketed dynamic padding |
| Optimiser | AdamW (β = 0.9, 0.95), weight decay 0, grad clip 1.0 |
| Learning rate | 1e-4 peak, linear warmup (~3%), cosine decay to 1e-5 |
| Precision | bf16 autocast |
| Hardware / time | 1 × RTX 3060 Laptop 6 GB, about 1 hour (~14.7k tokens/s) |
Code: scripts/sft.py
Evaluation
| Model | Answer loss | Answer perplexity |
|---|---|---|
| Hanuman-hi-42M (base, same prompt template) | 4.068 | 58.4 |
| Angad-hi-42M | 2.83 | 16.9 |
Measured on the 380 held-out validation pairs, with loss on answer tokens only. Qualitative checks are the sample answers above. Not yet evaluated on Hindi benchmarks such as IndicGLUE, IndicXNLI or IndicQA.
Architecture
Llama-style decoder (LlamaForCausalLM): hidden 512, FFN 1408, 8 layers, 8 heads, RoPE, RMSNorm, SwiGLU, tied embeddings, 32k SentencePiece Unigram vocabulary (about 1.1 tokens per Hindi word). Identical to the base model.
Bias, risks and limitations
- Hallucination: confident, fluent, wrong answers are common. Never rely on its facts.
- Data quality: machine-translated training data passes on translation errors and Western-centric content.
- No safety alignment: no RLHF or DPO, and no refusal training. It may produce harmful or biased content if asked.
- Single-turn, 512-token context.
Recommendations
Use it for learning and experiments only. For a useful Hindi assistant, start from a larger base model (the series plans Jambavan, ~110M) and curate native Hindi instruction data.
Environmental impact
| Hardware | 1 × RTX 3060 Laptop GPU |
| SFT time | about 1 hour (plus about 8–16 hours to pretrain the base model) |
| Energy | under 0.2 kWh for SFT (estimated) |
Model card authors and contact
ashishpatel26. Issues: https://github.com/ashishpatel26/Indian-Language-llm-model/issues
Citation
@misc{angad_hi_42m,
title = {Angad-hi-42M: an instruction-tuned Hindi language model trained from scratch on a 6 GB GPU},
author = {ashishpatel26},
year = {2026},
url = {https://huggingface.co/ashishpatel26/angad-hi-42m}
}