Whisper Small Jiangyin

This is a fine-tuned version of openai/whisper-small for Jiangyin dialect (江阴话) automatic speech recognition from WuSutra.com.
Wusutra.com is a dialect crowdsourcing website which implements the entire ML workflow — including audio upload, model training, validation, and inference. You can upload your own audios and even trigger the training yourself on wusutra.com. If you have further questions, feel free to message me.

[!TIP]
⚡ Looking for a smaller and faster option?
Please use the LoRA adapter version.
It provides the same Jiangyin dialect fine-tuning with much smaller storage.

📊 Evaluation on 45 Jiangyin dialect phrases: Character Error Rate (CER)

Model Details

• Base model: openai/whisper-small
• Language: Jiangyin dialect (江阴话) - a Wu Chinese dialect
• Task: Automatic Speech Recognition (ASR)
• Training data: Custom dataset of Jiangyin dialect recordings
• Model size: 244M parameters

Usage

from transformers import WhisperForConditionalGeneration, WhisperProcessor

# Load model and processor
model = WhisperForConditionalGeneration.from_pretrained("jxue/whisper_small_jiangyin")
processor = WhisperProcessor.from_pretrained("jxue/whisper_small_jiangyin")

# Transcribe audio
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
generated_ids = model.generate(inputs.input_features)
transcription = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

Training Details

• Training epochs: 15
• Batch size: 1
• Learning rate: 5e-6
• Training samples: 168
• Validation samples: 61
• Training infrastructure: AWS SageMaker ml.p3.2xlarge
• Training time: 15 mins

Performance

• Character Error Rate (CER): 0% on validation set
• Validation examples: See validation_report.json in model files

Limitations

• Optimized specifically for Jiangyin dialect

Significant improvement observed after fine-tuning on 119 dialect audio samples.

✅ Correct recognition example