Whisper Large-V3 for Sorani Kurdish (ckb) - V2

This is V2 of the fine-tuned Whisper Large v3 model for Central Kurdish (Sorani / ckb) Automatic Speech Recognition.

This version is trained on a significantly larger dataset compared to V1.

📐 Key Improvements in V2

• Trained with more data (20k+ audio samples: 4<=15 seconds)
• Better real-world performance on conversational and podcast-style audio

📊 Evaluation Results (Final)

• Word Error Rate (WER): 40.5%
• Evaluation Loss: 0.1668

🦻 Why V2 has higher WER than V1?

V2 shows a higher Word Error Rate (40.5%) compared to V1 (31.5%) mainly because of the evaluation dataset.

V2 was evaluated on a much larger and significantly harder test set, which includes strong regional dialects such as Hewlêr and Mukriyanî.

These dialects contain heavier pronunciation variations that are more challenging for ASR models.

Real-World Performance

Despite the higher validation WER, V2 performs very well on real-world audio.

_In manual testing across different audio scenarios:_

When speakers used clear and standard Sorani (academic pronunciation), the model produced correct Kurdish text in every single test.

Compared to Google Gemini, V2 was often more accurate in transcribing what was actually spoken.

Gemini sometimes applies auto-correction and guesses words instead of transcribing the audio literally.

For example, in Mukriyanî dialect, the word “Yeketi” is pronounced “Yecheti”, V2 correctly transcribed it as heard, while Gemini sometimes corrected it to the standard spelling.

demo_01.wav : " لە یەچی چاری دوهەزار و حەفتە دەستن بەو ئیشەی کرد، تا قیبان هەفت مانگ دەیبیتن دەستن پێکردییە و حەفتی حەفتی دوهەزار و حەفتەش یەکەم ئیشن بوو کە لەگەر ستاپ بیکەم و سیویەکی مانگیش ئینشاڵا سیویەکی داماناییە فیستیفالەکی لە ناو بازار بکەین بۆ روژی هەلووین

"

demo_02.wav : " رۆژانا هاوڵاتیان بە وێنەگرتنی فۆتۆ مودێل و کاری سینەمایی ئەنجامی دەدەن و ئیلافیش یەکێکە لەو گەنجانەیەکە لە شاری هەولێر کاری ماکیاژ کردنی ترسناچی س ایف ئێکس بۆ گەنجان و ئەکتەران ئەنجام دەدات "

demo_03.mp3 : " لێرا لە چین، مروڤیاتی خەریکی دروستکردنی گەورەتین چاوی دەسکردە دەسگای هێپس کە تیشکی ئەکس بەرهەم دێنێت یەک تریلیۆن جار لە بریقی خۆڕ بەیسترە لەنێوە ئەم پڕۆژە جیانیەدا، دابانو برووە دوو خوێنکاری کوردن کە خەریکی لەکولینەوەن لە نهێنیەکانی گردون و وزەی تیشکە گاماییەکان

"

⚠️ Important Usage Note

Due to the shared Arabic-based script and overlapping vocabulary, this model was trained using the persian language token as a forced decoder ID. This prevents English hallucinations and ensures the correct Kurdish script characters (like ە, ۆ, ێ, ڵ, ڕ) are generated. You must pass language="persian" during inference.

🚀 How to use

import torch
import librosa
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq

# Clone the repo
MODEL_PATH = "./whisper-large-v3-sorani-kurdish-ckb-v2"
TEST_AUDIO = "./demo_01.wav"

print("Loading model directly via PyTorch...")
processor = AutoProcessor.from_pretrained(MODEL_PATH)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    MODEL_PATH,
    torch_dtype=torch.float16,
    device_map="cuda"
)

print(f"Processing {TEST_AUDIO}...")
# Load audio and force 16kHz
audio, sr = librosa.load(TEST_AUDIO, sr=16000)

# Convert to input features
inputs = processor(audio, return_tensors="pt", sampling_rate=16000).to("cuda", dtype=torch.float16)

# Force the Persian/Kurdish script
forced_decoder_ids = processor.get_decoder_prompt_ids(language="persian", task="transcribe")

print("Generating text...")
# Generate with parameters that prevent the English repetition loop
generated_ids = model.generate(
    input_features=inputs.input_features,
    forced_decoder_ids=forced_decoder_ids,
    num_beams=5,          # Higher beam search = better accuracy
    do_sample=False,      
    length_penalty=1.0,   # Prevents the model from getting stuck in loops
    max_new_tokens=400    # Limits output length
)

# Decode the result
text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

print("\n" + "="*40)
print("FINAL TRANSCRIPTION:")
print("="*40)
print(text)
print("="*40)

📅 Roadmap (V3)

• More high-quality data
• Collecting clean recordings in other dialects