🌻 Sunflower-14B Model Card

Model Description

🌻 Sunflower-14B is a multilingual language model developed by Sunbird AI for Ugandan languages. Built on the Qwen 3-14B architecture, the model supports translation and text generation across 31 Ugandan languages plus English.

The latest version incorporates a GRPO fine-tuned checkpoint that significantly improves:

• Translation accuracy and fluency
• Factual question-answering
• Instruction following
• Response consistency and reasoning quality
• Reduced hallucinations compared to earlier releases

The updated model is especially stronger at multilingual factual Q&A, cross-lingual understanding, and producing more natural translations across low-resource Ugandan languages.

Developed by: Sunbird AI
Model type: Causal language model
Base architecture: Qwen3-14B
Languages: 31 Ugandan languages + English (see language codes above)

---

What's New in This Release

🚀 GRPO Fine-Tuned Version

This release integrates a GRPO (Group Relative Preference Optimization) fine-tuning stage through the checkpoint:

The GRPO optimization focused on:

• Improving factual correctness
• Better multilingual alignment
• More reliable instruction following
• Reducing repetitive or inconsistent outputs
• Improving answer grounding and coherence

Compared to the previous version, this release provides:

• More accurate translations
• Better conversational quality
• Stronger factual Q&A performance
• Improved multilingual reasoning

---

Intended Uses

Primary Use Cases

• Translation between English and Ugandan languages
• Translation between Ugandan languages
• Text generation in Ugandan languages
• Multilingual factual question answering
• Educational and knowledge-access applications
• Summarization and explanation tasks

---

Example Usage

import transformers
import torch

MODEL_PATH = "Sunbird/Sunflower-14B"

SYSTEM_MESSAGE = (
    "You are Sunflower, a multilingual assistant made by Sunbird AI "
    "who understands Ugandan languages and English. "
    "You specialise in accurate translations, factual question answering, "
    "summaries, explanations, and multilingual reasoning."
)

tokenizer = transformers.AutoTokenizer.from_pretrained(MODEL_PATH)

model = transformers.AutoModelForCausalLM.from_pretrained(
    MODEL_PATH,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

instruction = "Translate from Luganda to English: Wano webawaaba?"

messages = [
    {"role": "system", "content": SYSTEM_MESSAGE},
    {"role": "user", "content": instruction}
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer([prompt], return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=500,
    temperature=0.5,
    top_p=0.9,
    do_sample=True,
)

response = tokenizer.decode(
    outputs[0][len(inputs["input_ids"][0]):],
    skip_special_tokens=True
)

print(response)

---

API Access

For production use, the model is available through the Sunbird AI API:

https://api.sunbird.ai/

---

Training Details

Training Data

The model was trained on approximately 750 million characters of multilingual text collected from:

• Digitized books and educational materials
• Radio transcripts (500+ hours transcribed)
• Web data from MADLAD-400 and Common Crawl
• Existing multilingual datasets (SALT, FLORES-200, MT560, TICO-19)
• Dictionaries, proverbs, and cultural documents

---

Training Procedure

Stage 1: Continued Pretraining

• Base model: Qwen3-14B
• Objective: Next-token prediction
• Infrastructure: 4× H200 GPUs
• Configuration: DeepSpeed ZeRO-3

Stage 2: Supervised Fine-Tuning (SFT)

• Dataset: Instruction-response multilingual datasets
• Tasks:
  • Translation
  • Summarization
  • Question answering
  • Instruction following

Stage 3: Preference Optimization

• Initial alignment with preference optimization methods
• Reduced hallucinations and glitching behavior

Stage 4: GRPO Fine-Tuning (New)

• Checkpoint: jq/sunflower-14b-grpo-combined
• Objective:
  • Improve factual correctness
  • Improve multilingual reasoning
  • Improve answer quality and consistency
  • Better conversational alignment
• Outcome:
  • More accurate factual responses
  • Better translation fluency
  • Stronger instruction adherence
  • Improved multilingual Q&A quality

---

Evaluation

Translation Performance

The model was evaluated on a multilingual benchmark dataset covering practical scenarios including:

• Healthcare
• Agriculture
• Education
• Banking
• Government services

Average scores across 31 languages:

• chrF (xx→eng): 0.419
• chrF (eng→xx): 0.366
• BLEU (xx→eng): 19.61

The GRPO fine-tuned release additionally improves:

• Translation fluency
• Context preservation
• Answer consistency
• Factual reliability

---

Comparison with Previous Release

---

Comparison with Other Models

Limitations

• Performance varies across languages depending on available training data
• Some low-resource languages may still produce inconsistent outputs
• The model may still hallucinate in highly specialized domains
• Not intended for fully autonomous high-risk decision making
• Human review is recommended for medical, legal, or financial use cases

---

Bias and Ethical Considerations

The model was trained on multilingual web and cultural datasets which may contain historical or social biases. Users should:

• Review outputs in critical applications
• Validate sensitive translations with native speakers
• Be aware that low-resource language generation remains an active research area

---

Citation

@misc{sunflower2025,
  title={Sunflower: A Regional Approach to Large Language Models for Ugandan Languages},
  author={Akera, Benjamin and Nafula, Evelyn and Yiga, Gilbert and Natukunda, Phionah and Nsumba, Solomon and Muhanguzi, Joel and Namara, Janat and Sekalala, Imran and Walukagga, Patrick and Bainomugisha, Engineer and Mwebaze, Ernest and Quinn, John},
  year={2025},
  publisher={Sunbird AI}
}

---

Model Card Contact

For questions or issues, contact:

info@sunbird.ai