manthilaffs/Gamunu-4B-Instruct-Alpha

Gamunu-4b-Instruct-Alpha

සිංහල instruct LLM — Experimental Release

Gamunu-4b-Instruct-Alpha is the first experimental checkpoint of the Gamunu Project, a Sinhala-centric bilingual Large Language Model. Built through continued pre-training on Sinhala-rich academic and domain-specific data, it's fine-tuned for instruction following, reasoning, and culturally grounded interactions.

⚠️ Alpha Notice
This is an experimental research model.
It demonstrates strong Sinhala fluency, reasoning, and broad NLP coverage — but is single-turn only and not yet RLHF-aligned for multi-turn dialogue.
Use for research, benchmarking, and controlled deployments — not production.

🧪 Live Demo

Now you can try Gamunu-4b-Instruct-Alpha instantly on Hugging Face Spaces for free 👇

🔗 Gamunu ZeroGPU Demo

---

⚡ Capabilities

🔤 Language & Reasoning

• Fluent, idiomatic Sinhala generation
• Robust Sinhala ↔ English bilingual understanding
• Solid mathematical reasoning (percentages, word problems, arithmetic)
• Logical, step-by-step reasoning in QA tasks
• Structured, concise, and context-aware responses

🎭 Roleplay & Instruction

• Accurate adherence to single-turn instructions
• Expert persona simulation (teacher, scientist, analyst, advisor)
• Balanced, formal, and culturally aware tone

🧩 Supported NLP Tasks

• Text generation & completion
• Summarization (educational / contextual)
• Translation (Sinhala ↔ English)
• Paraphrasing and rewriting
• Question answering (factoid + reasoning)
• Instruction-based classification
• Role-specific expert responses

---

🚫 Limitations

• No conversational memory
• Occasional factual drift
• No RLHF or safety tuning yet
• Reasoning quality may degrade with ambiguous prompts

---

🎯 Intended Use

Best for

• Research & evaluation of Sinhala LLMs
• Educational assistants and analytical Q&A
• Cultural, marketing, and academic content generation
• Benchmarking instruction following in low-resource languages

Not for

• Medical, legal, or financial decision-making
• Production systems requiring factual reliability
• Processing sensitive or personal data

---

🧩 Training Details

Phase 1 – Continued Pre-training (CPT)

Focused on enhancing Sinhala linguistic coverage and contextual understanding for semantic depth.

Phase 2 – Supervised Fine-tuning (SFT)

Fine-tuned on a custom Sinhala instruction dataset emphasizing reasoning, roleplay, and assistant-style behavior.

---

📋 Model Summary

---

🧩 Base Model License

This model was fine-tuned from Google Gemma 3 4B, distributed under the
Gemma Terms of Use.

All rights to Gemma 3 4B remain with Google LLC.
The Gamunu-Instruct-4B-Alpha weights, datasets, and training code are released by
Manthila Mallawa (The Gamunu Project) under the Apache 2.0 License.
Use of the base model remains subject to Google's policies.

---

💬 Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "manthilaffs/Gamunu-4B-Instruct-Alpha"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

# Sinhala prompt template
sinhala_prompt = """පහත දැක්වෙන්නේ යම් කාර්යයක් පිළිබඳ විස්තර කරන උපදෙසක් සහ එයට අදාළ තොරතුරු ඇතුළත් ආදානයකි. ඉල්ලූ කාර්යය නිවැරදිව සම්පූර්ණ කළ හැකි ප්‍රතිචාරයක් සපයන්න.
### උපදෙස:
ඔබ ගැමුණු (Gamunu) නම් AI සහායකයායි.
ඔබේ කාර්යය වන්නේ පරිශීලකයන්ගේ උපදෙස් නිවැරදිව පිලිපැදීම හා අසා ඇති ප්‍රශ්නවලට නිවැරදිව පිළිතුරු සපයමින් ඔවුන්ට සහය වීමයි.
### ආදානය:
{}
### ප්‍රතිචාරය:
{}"""

# Example input
user_query = "හෙලෝ ගැමුණු! මම සමන්, ඔයාට කොහොමද?"

prompt = sinhala_prompt.format(user_query, "")
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate
with torch.inference_mode():
    outputs = model.generate(**inputs, max_new_tokens=250)

# Decode and clean output
text = tokenizer.decode(outputs[0], skip_special_tokens=True)
if "### ප්‍රතිචාරය:" in text:
    text = text.split("### ප්‍රතිචාරය:")[-1].strip()

print(text)

---

🧾 How to Cite

If you use Gamunu-Instruct-4B-Alpha in your work, please cite as follows:

APA

Mallawa, M. (2025). Gamunu-Instruct-4B-Alpha: A Sinhala-centric bilingual instruction-tuned language model. The Gamunu Project. Retrieved from https://huggingface.co/manthilaffs/Gamunu-Instruct-4B-Alpha

BibTeX

@misc{mallawa_gamunu_instruct_4b_alpha_2025,
  author       = {Mallawa, Manthila},
  title        = {Gamunu-Instruct-4B-Alpha: A Sinhala-centric bilingual instruction-tuned language model},
  year         = {2025},
  publisher    = {The Gamunu Project},
  howpublished = {\url{https://huggingface.co/manthilaffs/Gamunu-Instruct-4B-Alpha}}
}