Whisper Large-V3 — Kenyan English (Non-Standard, Noise-Robust)

Fine-tuned from openai/whisper-large-v3 on non-standard Kenyan English speech, with waveform-level noise augmentation to improve robustness under real-world Kenyan field deployment conditions (mobile networks, ambient crowd noise, device volume variation).

Developed by Alexander S. Mainye.

Upgrade from whisper-small: this model improves on smainye/whisper-small-kenyan-english-nonstandard-v2 across every metric — clean WER (15.77% → 13.65%), noisy WER (21.62% → 18.31%), and robustness gap (5.85 pp → 4.66 pp).

This model is published as: smainye/whisper-large-v3-kenyan-english-nonstandard-v1

Model Details

Training Report

Performance

Evaluated on the held-out test split of cdli/kenyan_english_nonstandard_speech_v1.0 (926 examples). Clean evaluation uses unaugmented audio; noisy evaluation applies the same waveform augmentation pipeline used during training.

The 4.66 pp robustness gap is the best result across the entire whisper-small and whisper-large-v3 fine-tuning series on this dataset, representing the first time noisy test WER fell below 20% for this domain.

Comparison with whisper-small-kenyan-english-nonstandard-v2

Training Details

Dataset

cdli/kenyan_english_nonstandard_speech_v1.0 — non-standard Kenyan English speech covering etiologies.

Fine-Tuning Strategy

Full fine-tuning with all 1.55B parameters trainable, including the encoder. Prior experiments (frozen encoder) confirmed that unfreezing the encoder was critical for noise robustness — it enables the acoustic feature extraction layer to adapt directly to Kenyan field conditions rather than relying on the decoder alone.

Augmentation Pipeline

Waveform-level augmentation applied during training only (AUGMENT_PROB = 0.4). Evaluation always used clean audio.

SpecAugment was also applied during training (mask_time_prob = 0.05, mask_feature_prob = 0.05).

Hyperparameters

Training ran the full 2,000-step budget without early stopping triggering. The best validation WER (19.24%) was achieved at step 1,800 — consistent with the polynomial scheduler's gradual decay allowing late-stage improvement.

Usage

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import torch

processor = WhisperProcessor.from_pretrained("smainye/whisper-large-v3-kenyan-english-nonstandard-v1")
model = WhisperForConditionalGeneration.from_pretrained("smainye/whisper-large-v3-kenyan-english-nonstandard-v1")

# Load your audio (must be 16kHz, mono)
# audio is a numpy array of shape (samples,)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")

with torch.no_grad():
    predicted_ids = model.generate(
        inputs["input_features"],
        language="en",
        task="transcribe"
    )

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription[0])

Notes on intended use

• Optimised for non-standard Kenyan English.
• Most robust to ambient crowd noise and GSM/mobile codec artifacts (as trained).
• Not fine-tuned for standard American or British English; general-purpose WER on those varieties may differ from openai/whisper-large-v3 baseline.
• Evaluation was conducted on a noise-augmented test set simulating field conditions; real-world WER will vary based on recording quality and acoustic environment.
• This model is substantially larger than whisper-small (1.55B vs 241M parameters) and requires more memory at inference time. For resource-constrained environments, consider smainye/whisper-small-kenyan-english-nonstandard-v2 instead.

Limitations

• Dataset size: the training set comprises 4,243 examples. Performance on speaker demographics, dialects, or acoustic conditions not well-represented in cdli/kenyan_english_nonstandard_speech_v1.0 may degrade.
• Room reverb not covered: reverb-heavy environments (lecture halls, large rooms) were not part of the augmentation pipeline. Performance in strongly reverberant conditions is untested.
• Inference cost: at 1.55B parameters, this model requires significantly more compute at inference time than whisper-small. Plan accordingly for production deployments.
• Language: English only. Not suitable for Swahili or other Kenyan languages.

Training Infrastructure

Training was conducted using Modal GPU infrastructure (NVIDIA A100-SXM4-80GB) with the Hugging Face transformers Seq2SeqTrainer.

Citation

If you use this model in your work, please cite it as:

@misc{whisper-large-v3-kenyan-english-nonstandard,
  author       = {Mainye, Alexander Sheldon},
  title        = {Whisper Large-V3 Fine-tuned on Non-Standard Kenyan English Speech},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/smainye/whisper-large-v3-kenyan-english-nonstandard-v1}
}