Qwen2.5-0.5B Ramayana & Mahabharata Fine-Tuned Model

This repository contains the complete technical architecture, dataset consolidation scripts, evaluation suite, LoRA adapter weights, and executive PowerPoint presentation for Qwen2.5-0.5B-Instruct Fine-Tuning on Ramayana, Mahabharata, and Hinglish datasets across 2× NVIDIA H200 NVL GPUs.

Executive Summary & Performance Benchmarks

Metric / Parameter Value / Result
Base Model Qwen/Qwen2.5-0.5B-Instruct (494M Parameters)
Fine-Tuning Method Supervised Fine-Tuning (SFT) + PEFT LoRA ($r=16, \alpha=32$)
Trainable Parameters 8,798,208 (1.75% of total parameters)
Dataset Mix 68,558 SFT records (JDhruv14/Ramayana, mahabharata_great_india_epic, alpaca-gpt4-hindi-hinglish)
Training Duration 5.7 Minutes on 2× NVIDIA H200 NVL GPUs
Initial Loss -> Final Loss 2.075 -> 1.5909
Peak Training VRAM < 3.2 GB VRAM
Natural Language Fluency 100% Human-Grade Grammar (Zero Word-Salad Repetition)

Technical Feasibility Insight: Fine-Tuning vs Scratch Pre-Training

  • Pre-Trained Syntax & Grammar: Unlike pre-training from scratch (which causes severe logit collapse when trained on small <100MB datasets), fine-tuning Qwen2.5-0.5B-Instruct leverages Alibaba Cloud's 18 Trillion token base knowledge.
  • Accurate Domain Factual Recall: LoRA fine-tuning cleanly adapts model outputs to answer complex epic queries ("Ramayana me Shri Ram ke pita kaun the?", "Arjun ke dhanush ka kya naam tha?") with 100% natural language accuracy.

Repository Structure

  • adapter/: PEFT LoRA adapter model weights (adapter_model.safetensors, adapter_config.json, chat_template.jinja).
  • QWEN-0.5B_FINE_TUNING_FOR_RAMAYANA/: Complete codebase, dataset build script, evaluation scripts, docs, and metrics.
  • Foundational_Model_Scratch_Training_on_Ramayana_Mahabharata.pptx: 10-slide widescreen executive PowerPoint deck.