High-Fidelity Kashmiri Whisper Large-V3-Turbo Transcribe Model (LoRA)

Backbone Architecture: openai/whisper-large-v3-turbo (809M parameters)
Task: Kashmiri Perso-Arabic Automatic Speech Recognition (ASR)
Adapter Format: PEFT LoRA (r=64, alpha=128, dropout=0.05 across q_proj, k_proj, v_proj, out_proj, fc1, fc2)
Evaluation CER: 22.42%
Evaluation Loss: 0.5968 (Train Loss: 0.6382)
Runtime: 26,356 seconds (~7.32 hours) over 8 full epochs on an NVIDIA Tesla P100 GPU.

---

1. Model Overview & Kashmiri Conditioning

This model is fine-tuned to transcribe continuous Kashmiri speech into authentic Kashmiri Perso-Arabic orthography, preserving native Kashmiri vowels (ـٕ, ٲ, ۄ, ۆ, ؠ).

Training Datasets:

1. gopikrishnan2005/kashmiri-text-to-speech: Studio-quality phonetically balanced native Kashmiri speech recordings.
2. umar1103/converteddatanew: Natural Kashmiri conversational speech utterances.

---

2. Key Performance Metrics

---

3. How to Transcribe Kashmiri Audio

import torch
from transformers import WhisperProcessor, WhisperForConditionalGeneration
from peft import PeftModel
import soundfile as sf
import librosa

BASE_MODEL = "openai/whisper-large-v3-turbo"
ADAPTER_REPO = "Sameer0313/kashmiri-whisper-lora"

# 1. Load Processor & Base Architecture
processor = WhisperProcessor.from_pretrained(BASE_MODEL, language="urdu", task="transcribe")
base_model = WhisperForConditionalGeneration.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

# 2. Attach Kashmiri LoRA Adapter
model = PeftModel.from_pretrained(base_model, ADAPTER_REPO)
model.eval()

# 3. Transcribe 16kHz Audio
def transcribe_kashmiri(audio_path: str) -> str:
    speech, sr = librosa.load(audio_path, sr=16000)
    input_features = processor(speech, sampling_rate=16000, return_tensors="pt").input_features
    input_features = input_features.to(model.device, dtype=model.dtype)
    
    with torch.no_grad():
        predicted_ids = model.generate(
            input_features,
            num_beams=2,
            max_new_tokens=225
        )
    return processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]

---

4. Citation & Acknowledgments

Part of the open-source Kashmiri computational linguistics and speech intelligence project at sameerahmad86232/cipher.