Yama: zero-shot text classifier for Indian languages and Hinglish

Yama (yama-indic) is an open-weights, Apache-2.0 zero-shot text classifier for Hindi, Hinglish, 10 other Indian languages and English. You pass a text and plain-English questions with your own labels; it returns a calibrated probability for every label in one forward pass, without fine-tuning. It is a 586M-parameter encoder (XLM-RoBERTa-large), not a generative LLM, so answers are fast, deterministic and cheap to run.

📖 Write-up: Before You Invent a New Architecture, Try a Bigger Encoder (Towards AI) · Summary and FAQ · By Arun Prabhakaran · @LatentAILabs

Same test samples and harness for every model; details under Comparison with other zero-shot classifiers.

Use it for: intent detection and routing, support-ticket triage, sentiment, topic and news classification, yes/no checks ("is this a complaint?") and 1–N ratings ("how urgent?"), on text in Hindi (Devanagari or romanised), Hinglish, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Urdu, Odia and English. Labels, questions and the number of options are defined at inference time, so one model handles many tasks.

How it works

Yama is a typed-decision model. You give it a text (a message, ticket, review, headline or document) and one or more questions written in plain English. It answers every question in a single forward pass, with a calibrated probability for each option. Question schemas are defined at inference time, so it handles new labels and new tasks without retraining.

The name. In Indian tradition Yama is also Dharmaraja, the judge who weighs what is placed before him and gives a verdict. This model does the same with the options you give it.

Successor to laya-indic. Yama is v3 of LatentAILabs/laya-indic, released as a separate model because the encoder changed. The v1 and v2 (mmBERT-base) weights stay in that repository.

Yama is an independent personal release. It is not affiliated with or endorsed by HAL-X AI. It uses the THX-01 decision architecture and the open-source thx01 library (Apache-2.0, HAL-X AI), but it is built on the FacebookAI/xlm-roberta-large encoder (MIT), not on THX-01's weights. All training data is public and permissively licensed, plus labels generated and verified by open-weights models (Qwen3, Qwen3.6, Gemma 4). As far as can be verified, no proprietary-model output was used. The one exception that can't be fully checked is NVIDIA's Nemotron Safety Guard v3, which is partly machine-translated and synthetic and doesn't name the models used; see LICENSES.md.

What's new in v3

• A larger encoder. v3 is built on XLM-R-large (560M) instead of mmBERT-base. Overall accuracy rises from 0.800 to 0.862 and mean calibration error falls from 0.091 to 0.070. The biggest gains are in Indian languages: SIB-200 Indic topic 0.670 → 0.844, IndicSentiment 0.884 → 0.950, MASSIVE Indic 0.849 → 0.893.
• Causal reasoning is no longer at chance. IndicCOPA rises from 0.513 to 0.624. Telugu and Gujarati, which had no cause/effect training data at all, improved as much as the other languages (0.528 → 0.634, 0.529 → 0.632).
• More real Indian-language text. 54,198 new questions about real Sangraha text, written by Gemma 4 and verified by Qwen3.6, with extra weight on Odia and Urdu, which v2 barely covered.
• Trade-offs. About twice the compute per decision of v2, and the input limit is 512 tokens instead of 1,024. Banking77 (0.819 → 0.791) and SMS spam (0.964 → 0.944) are lower than v2. BoolQ-hi is unchanged (0.618 → 0.616).

Question types

THX-01 also supports number, excerpt and "cite": true. This model was not trained for those and does not support them.

Quickstart

import thx01   # pip install thx01

agent = thx01.load("LatentAILabs/yama-indic")          # or a local directory; GPU if available
# v2 (previous model): thx01.load("LatentAILabs/laya-indic")

text = "Kal se mera UPI payment fail ho raha hai, paise kat gaye par merchant ko nahi mile"
out = agent.system_one(text, {
    "topic":  {"type": "choice", "instructions": "What is the customer writing about?",
               "criteria": {"failed_payment": "payment failed or money debited but not received",
                            "refund": "asking for a refund", "account": "login or account access", "other": None}},
    "urgent": {"type": "noul", "instructions": "Does the customer need help urgently?"},
    "anger":  {"type": "score", "instructions": "How frustrated is the customer?",
               "criteria": ["calm", "mildly annoyed", "clearly frustrated", "furious"]},
})
# topic  -> failed_payment (0.96)
# urgent -> P(yes) 0.74
# anger  -> "clearly frustrated" 0.88

The response's model field says "THX-01". That is the thx01 library's default label; the weights are this model's.

For many texts, use agent.predict_batch(texts, questions, batch_size=64). questions can be one schema shared by every text or a list with one schema per text.

Write questions in English and leave the text in its own language and script. That is how the model was trained.

Benchmarks

Zero-shot accuracy on 67 tasks, compared with THX-01 (322M, same architecture, roughly 2.1M training decisions) and with v2 (laya-indic v2). All models were scored with the same harness, the same prompts and the same test samples. The harness is adapted from THX-01's open benchmark code (bench/tasks.py).

Per task: v3 is ahead of THX-01 on 63 of 67 tasks, behind on 3, tied on 1, and ahead of v2 on 60, behind on 6, tied on 1.

Comparison with other zero-shot classifiers

Four widely used zero-shot classifiers (the Hub's most-downloaded one, bart-large-mnli, which is English-only, and three multilingual ones), scored on the same test samples with the same harness. Only choice tasks are compared (60 tasks), because NLI models have no native yes/no-question or rating mode. For the NLI models each option becomes the hypothesis "This text is about {option description}." (COPA alternatives are used as-is), and probabilities are a softmax over entailment scores, as in the Hugging Face zero-shot-classification pipeline.

Full model ids: facebook/bart-large-mnli, MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7, joeddav/xlm-roberta-large-xnli, MoritzLaurer/bge-m3-zeroshot-v2.0.

• On the 33 tasks not in its training data, Yama is ahead of bart-large-mnli, mDeBERTa-xnli and xlm-roberta-large-xnli on every task, and ahead of bge-m3-zeroshot-v2.0 on 29, behind on 3 (AG News, DAIR Emotion, IndicCOPA Marathi) and tied on 1. Mean calibration error on these tasks is 0.092 for Yama and for bge-m3-zeroshot-v2.0, 0.100 for bart-large-mnli, 0.140 for xlm-roberta-large-xnli and 0.223 for mDeBERTa-xnli.
• The bottom row (MASSIVE, Banking77, Hinglish-TOP) is shown for completeness only: Yama was trained on those datasets' training splits and the NLI models were not.
• Sanity check: our harness gives the NLI models scores at or above their own published results (for example bart-large-mnli on AG News: 0.785 accuracy here, 0.703 macro-F1 on the bge-m3-zeroshot card), so the prompt wording is not handicapping them.
• † The bge-m3-zeroshot-v2.0 card reports for AG News and DAIR Emotion only the score of a run trained on up to 500 examples per class of those datasets (0.892 and 0.697 F1), and our scores match those, so its AG News and Emotion results are probably not zero-shot.
• Yama's own synthetic training data includes generic topic, sentiment and emotion questions (not these datasets), so "not in training data" means the dataset, not the task type.

Read these numbers with the training data in mind. None of the models saw these test sets. Some tasks, however, have their training splits or similar task formats in one model's data and not in the other's:

All training texts, including the Sangraha corpus the real-text questions were cut from (line by line), were deduplicated against every evaluation set, using normalised exact match.

Per-task results (accuracy and expected calibration error)

ECE is expected calibration error with 15 equal-width bins; lower is better. Mean ECE over the 67 tasks is 0.070 for v3, 0.091 for v2 and 0.097 for THX-01.

‡ BoolQ-hi is left out of every mean, as in earlier versions. Our harness had a label-parsing bug in the first runs of this benchmark; all rows here were scored after the fix.

Where v3 is stronger than THX-01 and v2:

• Indian-language topic classification (SIB-200 Indic +0.21 over THX-01, +0.17 over v2) and sentiment (+0.07 over THX-01).
• Low-resource languages THX-01 also covers: Georgian, Kazakh, Uzbek, Azerbaijani, Turkish.
• Causal reasoning in Indian languages (IndicCOPA +0.10), including the two languages it had no data for.
• New decision schemas it has never seen (synthetic held-out +0.21 over THX-01).

Where it is weaker:

• AG News (0.878 vs 0.931 for THX-01). THX-01 reports training on public benchmarks' training splits; AG News is excluded here.
• Banking77 and SMS spam are a little lower than v2.
• BoolQ-hi (0.616) is below THX-01 (0.647); passage yes/no questions remain hard.
• IndicCOPA (0.624) is well above chance but far from strong.

Training

Method. RLCD (Reinforcement Learning for Calibrated Decisions), as introduced by THX-01 and implemented in thx01:

• Policy-gradient term: sampling from noisy logits, rewarded by a strictly proper scoring rule.
• Soft cross-entropy term: on label-smoothed targets.
• Calibration: a temperature for each question type, fitted after training on held-out data.

Decision training starts from FacebookAI/xlm-roberta-large with a freshly initialised decision head. (Continued pretraining on Sangraha, which helped v2, gave no measurable gain on XLM-R-large, so v3 does not use it.)

Public data (285,078 decisions; unchanged from v2; every source is CC0, CC-BY, Apache-2.0 or MIT; full attribution in LICENSES.md):

Model-generated data (306,139 verified questions). Every question was answered blind by a second, different open-weights model and kept only if the two agreed.

All five models are open-weights and Apache-2.0. The v3 labels were generated on our own GPU with vLLM. The three v3 parts are published as LatentAILabs/yama-indic-gemma (configs real_text, copa, boolq, with held-out test splits).

• Synthetic decisions (v1): the generator proposes a schema, gold answers are sampled first, then it writes a text that makes them true, in a chosen language, script and register. About 68% of texts are in Indian languages or Hinglish. Published as LatentAILabs/laya-indic-synthetic; see SYNTHETIC_DATA.md.
• Real-text questions (v2 and v3): 1–3 sentence snippets cut from Sangraha "verified" documents (AI4Bharat, CC-BY-4.0). The generator picks a topic from a fixed 16-topic list and writes 2–3 snippet-specific questions; topic options are assembled in code, and in 7% of cases the correct topic is removed so "none of the options" is the answer. The v2 part is published as LatentAILabs/laya-indic-real-text. The v3 part covers all 11 languages roughly equally, with extra Odia (2,324 texts) and Urdu (2,212).
• Cause/effect questions (v3): Gemma reads a real snippet and writes a premise, a "what caused this?" or "what happened next?" question and two alternatives, in the snippet's language. Hindi, Marathi, Bengali, Tamil, Kannada, Malayalam, Punjabi, Odia and Urdu; not Telugu or Gujarati. Only 58% of generated questions passed verification.
• Passage yes/no questions (v3): a longer real snippet is the passage; Gemma writes yes/no questions about it in the passage language, about half answered "no". Gujarati, Tamil, Telugu, Bengali, Marathi and Kannada; not Hindi.

Limitations

• Text classification only. It does not do number, excerpt or citation questions, open-ended generation or multi-step reasoning. IndicCOPA (0.62) shows commonsense causal reasoning is still weak.
• Instructions in English. Questions and option descriptions should be in English; the text can be in any supported language.
• Long texts are truncated at 512 tokens per question (v2 allowed 1,024). XLM-R's tokenizer is 25–40% more compact on Indian scripts than v2's, which offsets part of this.
• Calibration is good on average but not everywhere. IndicCOPA and BoolQ-hi are poorly calibrated (ECE above 0.2). Recalibrate on your own data before using probabilities as thresholds.
• About 52% of the training labels come from language models (Qwen3, Gemma 4) and were checked only by another language model. Synthetic Indic text is cleaner and more formal than real user text. Sangraha is mostly news and web prose, not chat.
• Content warning. The safety and toxicity datasets (Aegis 2.0, Nemotron Safety Guard, Measuring Hate Speech, Civil Comments, Aya Red-teaming) contain offensive and harmful text. The model can classify such content but is not a tested content-moderation system.

FAQ

What is the best zero-shot classifier for Hindi and Hinglish?

In our tests Yama is the most accurate open zero-shot classifier for Indian languages. On 22 Indian-language tasks that no model was trained on (SIB-200 topic, IndicSentiment, IndicCOPA), it scores 0.813 accuracy, against 0.753 for bge-m3-zeroshot-v2.0, 0.586 for xlm-roberta-large-xnli, 0.532 for mDeBERTa-v3-base-xnli and 0.347 for bart-large-mnli. On Hinglish voice-assistant requests (Hinglish-TOP) it scores 0.942, but it was trained on that dataset's training split. Test it on a sample of your own data before relying on it.

Which languages does Yama support?

Instructions are written in English; the text can be in Hindi (Devanagari or romanised), Hinglish, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Urdu, Odia or English. It was also trained on Russian, Turkish, Azerbaijani, Kazakh, Uzbek, Georgian, Persian, Arabic, German, French, Spanish, Chinese, Japanese and Korean, and inherits XLM-RoBERTa's coverage of about 100 languages, but it is tested most on the languages above.

How is Yama different from bart-large-mnli or other NLI zero-shot models?

NLI models score each label separately as "does this text entail this text is about X?". Yama reads the question and all the options together and was trained directly on typed decisions, so it also answers yes/no questions and ordinal ratings, not just pick-one-label, and its probabilities are calibrated. See the comparison table under Benchmarks.

How is it different from asking an LLM?

Yama does not generate text. One forward pass of a 586M encoder returns probabilities for every option, so there is no output parsing, no sampling and no prompt-format drift. A single decision takes about 17 ms on an NVIDIA H100 and about 120 ms on 8 CPU threads (AMD EPYC); batched, it handles about 1,400 decisions per second on one H100. It is not a replacement for an LLM on open-ended or multi-step reasoning tasks.

Can I use it commercially?

Yes. The weights are Apache-2.0. All training data is CC0, CC-BY, Apache-2.0 or MIT, with no ShareAlike or NonCommercial sources. No proprietary-model output was used as far as can be verified; Nemotron Safety Guard v3 doesn't name the models used to generate it. See LICENSES.md for attributions.

How do I make it more accurate on my task?

Write clear option descriptions (they matter more than the keys), keep questions in English, and recalibrate the probabilities on a few hundred of your own labelled examples before using them as thresholds.

What happened to laya-indic?

It is still available at LatentAILabs/laya-indic with the v1 and v2 weights (mmBERT-base encoder). Yama is its successor: v3 moved to a larger encoder (XLM-RoBERTa-large), so it was released as a separate model. Use Yama for new work; it scores 0.862 against 0.800 for v2 on the same 67 tasks.

Acknowledgements and licence

• Architecture, training objective and inference library: THX-01 / thx01 by HAL-X AI (Apache-2.0).
• Benchmark harness: THX-01's bench/tasks.py, including its MASSIVE scenario descriptions and label humaniser, which were also used in some training instructions.
• Encoder: XLM-RoBERTa large, by Facebook AI (MIT). v1 and v2 used mmBERT-base, by JHU CLSP (MIT).
• Indian-language corpus: Sangraha, by AI4Bharat (CC-BY-4.0), source of the real-text questions.
• Label models: Qwen3 and Qwen3.6 (Alibaba Qwen team) and Gemma 4 (Google DeepMind), all Apache-2.0.

Copyright 2026 Arun Prabhakaran. The model is released under the Apache License 2.0. See NOTICE for the required attributions and LICENSES.md for every dataset's authors and licence.

Citation

@misc{prabhakaran2026yama,
  title  = {Yama: a zero-shot typed-decision model for Indian languages and Hinglish},
  author = {Prabhakaran, Arun},
  year   = {2026},
  url    = {https://huggingface.co/LatentAILabs/yama-indic}
}

More from the author

Built by Arun Prabhakaran (Latent AI Labs). Open models for Indian languages:

• Chitragupta: PII detection and redaction
• Nandi: complaint routing
• Bhasha: word-level language ID

Write-up: Before You Invent a New Architecture, Try a Bigger Encoder · Summary and FAQ: https://latentailabs.github.io/yama-bigger-encoder/

• Articles and FAQs: latentailabs.github.io
• Medium: @arunp0
• YouTube: @LatentAILabs — how AI works, measured on real GPUs
• GitHub: arunp0 · LinkedIn: arunp0