Whisper Small — Arabic Smart-Home Voice Commands (LoRA)
A LoRA fine-tune of openai/whisper-small for transcribing short Levantine Arabic (Jordanian dialect) voice commands aimed at home automation — lights, AC, and shades/blinds.
Model Details
- Base model:
openai/whisper-small - Fine-tuning method: LoRA (PEFT) — base weights frozen, adapter trained on top
- Language: Arabic (Jordanian dialect)
- Task: automatic speech recognition (transcription), intended as the ASR stage of a voice-controlled smart-home pipeline
- License: MIT (inherited from the base model)
LoRA Configuration
| Parameter | Value |
|---|---|
r |
32 |
lora_alpha |
64 |
lora_dropout |
0.05 |
target_modules |
q_proj, v_proj |
Training Data
This is a pilot dataset — one speaker, one recording condition. 170 recordings from a single aptured near-mic in a quiet room. Recordings cover 3 device categories (lights, AC, shades) across 14 distinct actions (on/off/set/dim/brighten/open/close/increase/decrease/etc.), drawn from a setgned to capture natural phrasing variation for each command (e.g. multiple ways to say "turn offthe light").
Known limitation: because training data comes from one speaker and one acoustic condition, this model is expected to perform noticeably worse on other voices, accents, microphones, or noisy/distant recording conditions. A larger dataset (5 speakers, 300+ commands, varied conditions) is in progress and will replace tnce available — check the repo for updates.
Evaluation
Evaluated on a 26-sample held-out split (15%) from the same speaker/condition as training, normalvariance (diacritics, digit script) before scoring:
| WER | CER | |
|---|---|---|
Zero-shot whisper-small baseline* |
0.835 | 0.395 |
| This fine-tune | 0.106 | 0.037 |
How to Use
from transformers import WhisperForConditionalGeneration, WhisperProcessor
repo_id = "YahyaMujahed/whisper-small-arabic-smart-home"
model = WhisperForConditionalGeneration.from_pretrained(repo_id)
processor = WhisperProcessor.from_pretrained(repo_id)
# audio: a mono float32 numpy array at 16kHz
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
predicted_ids = model.generate(inputs.input_features)
transcript = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
Intended Use
Designed as the transcription stage of a small voice-controlled smart-home assistant — pair the ont-matching layer (keyword-based or LLM-based) to extract a structured action (device, action,value) for actual device control. Not intended as a general-purpose Arabic ASR model.
Out-of-Scope Use
Not evaluated on: other Arabic dialects, other speakers, noisy or far-microphone conditions, or fc outside the smart-home command domain.