whisper-base-ne-en-finetuned-v3
This model is a fine-tuned version of openai/whisper-base for Nepali and English code-switched speech recognition.
Model Details
- Base Model: openai/whisper-base
- Language: Nepali-English (code-switched)
- Task: transcribe
- Fine-tuned on: Custom Nepali ASR dataset
Training Configuration
- Epochs: 3
- Batch Size: 8 x 2 = 16 (effective)
- Learning Rate: 1e-05
- Mixed Precision: FP16
- Training Samples: 21700
- Validation Samples: 2712
- Test Samples: 2713
Evaluation Results
| Split | WER |
|---|---|
| Test | 43.34% |
Usage
from transformers import WhisperProcessor, WhisperForConditionalGeneration
import librosa
# Load model and processor
processor = WhisperProcessor.from_pretrained("Bijay13/whisper-base-ne-en-finetuned-v3")
model = WhisperForConditionalGeneration.from_pretrained("Bijay13/whisper-base-ne-en-finetuned-v3")
# Load audio
audio, sr = librosa.load("your_audio.wav", sr=16000)
# Transcribe
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features
predicted_ids = model.generate(input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)
License
This model is released under the Apache 2.0 license.