Annamacharya Sangeetha Sankeerthana AI (Sarvam-1 LoRA)

This repository contains a fine-tuned LoRA adapter based on the Sarvam-1 (2B) model, specialized to compose traditional, classical Telugu Sankeerthanas in the legendary style of saint-composer Tallapaka Annamacharya.

🌟 Model Capabilities

• Classical Telugu Structure: Generates songs featuring a traditional Pallavi, followed by multiple thematic Charanams.
• Signature Mudra Integration: Concludes stanzas organically utilizing the classical Annamayya signature ("శ్రీవేంకటేశుఁడు" / "శ్రీవేంకటపతి").
• Bilingual & Transliteration Support: Understands complex instructions in native Telugu script, English transliterated Telugu (Tanglish like: "padmavati ammavari paina oka pata rayi"), or raw English inputs via an automated translation gateway.

---

🛠️ Installation & Setup

To run inference using this model, you need to install the dependencies including unsloth for ultra-fast performance.

pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install torch transformers peft accelerate bitsandbytes deep-translator

---

🚀 Quickstart Inference Code

Option 1: Basic Telugu/Tanglish Inference

from unsloth import FastLanguageModel
import torch

# Load the base model and the custom LoRA adapter
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "nithin093/annamacharya_sangeetha_sankeerthanalu",
    max_seq_length = 2048,
    dtype = None,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

prompt = """Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
పద్మావతీ దేవి వైభవమును మరియు ఆమె కరుణను కీర్తిస్తూ ఒక మధురమైన అన్నమయ్య శైలి సంకీర్తన వ్రాయుము.

### Response:
"""

inputs = tokenizer([prompt], return_tensors = "pt").to("cuda")
outputs = model.generate(
    **inputs,
    max_new_tokens = 512,
    use_cache = True,
    temperature = 0.7,
    top_p = 0.9,
    repetition_penalty = 1.15
)

print(tokenizer.batch_decode(outputs, skip_special_tokens = True)[0])

---

Option 2: Automatic English Translation Pipeline

Perfect for applications serving end-users who wish to prompt in English while expecting high-quality classical Telugu output.

from deep_translator import GoogleTranslator

def generate_sankeerthana_from_english(english_instruction):
    # 1. Translate English prompt to Telugu
    telugu_translation = GoogleTranslator(source='en', target='te').translate(english_instruction)
    
    # 2. Format with the instruction-tuning template
    formatted_prompt = f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{telugu_translation}

### Response:
"""
    
    # 3. Tokenize and generate
    inputs = tokenizer([formatted_prompt], return_tensors = "pt").to("cuda")
    outputs = model.generate(
        **inputs,
        max_new_tokens = 512,
        use_cache = True,
        temperature = 0.7,
        top_p = 0.9,
        repetition_penalty = 1.15
    )
    
    generated_output = tokenizer.batch_decode(outputs, skip_special_tokens = True)[0]
    
    if "### Response:" in generated_output:
        return generated_output.split("### Response:")[1].strip()
    return generated_output

# Run the pipeline
english_prompt = "Write a classical sankeerthana about the divine beauty of Goddess Padmavati and her eternal love connection with Lord Venkateswara"
print(generate_sankeerthana_from_english(english_prompt))

---

📊 Training Specifications

• Base Model: sarvamai/sarvam-1
• Dataset: 3,000+ instruction-response pairs of clean, historically accurate Tallapaka Annamacharya Sankeerthanas (keerthanas_instructions_3k.jsonl).
• Framework: Unsloth & trl.SFTTrainer
• GPU: NVIDIA Tesla T4 (Colab Free Tier compatible)
• Hyperparameters:
  • Epochs: 1
  • Batch Size: 2 per device (Gradient Accumulation Steps: 4)
  • Optimizer: adamw_8bit
  • Learning Rate: 2e-4
  • LoRA Config: r=16, alpha=16, Target modules including Attention & MLP projections.

---

📝 License

This project is licensed under the Apache-2.0 License.