Whisper-small: Tunisian Derja (CTranslate2 / faster-whisper, int8)

This is the CTranslate2 export of TheBiggyBang/whisper-small-tunisian-derja. It is a Whisper-small fine-tune for Tunisian Arabic (Derja) with French/English code-switching, quantized to int8_float16 (250 MB).

Use it with faster-whisper:

• on GPU (int8_float16) or CPU (int8);
• with the included live_asr.py for low-latency microphone transcription.

Code, training notebook and full evaluation: GitHub

Results

Held-out test set: 482 clips (36.6 min) from all LinTO test splits, recording-disjoint from training and checkpoint selection.

• RTF = processing time / audio length. GPU numbers are from a single T4; CPU numbers are from a 4-vCPU Kaggle machine.
• Normalized WER removes Derja spelling variants that are not recognition errors (أ/إ/آ→ا, ى→ي, ة→ه, digits, punctuation, Latin case, standalone «و»).

How to use

pip install faster-whisper

from faster_whisper import WhisperModel

model = WhisperModel("TheBiggyBang/whisper-small-tunisian-derja-ct2",
                     device="cuda", compute_type="int8_float16")   # CPU: device="cpu", compute_type="int8"

segments, _ = model.transcribe("audio.wav", language="ar", task="transcribe", beam_size=3,
                               condition_on_previous_text=False, vad_filter=True)
print(" ".join(s.text.strip() for s in segments))

• language="ar" matters: the model was fine-tuned with the Arabic language token.
• The reported scores used beam_size=3, condition_on_previous_text=False, without_timestamps=True and no VAD on pre-segmented clips. vad_filter=True is recommended for long recordings.

Live microphone transcription (live_asr.py)

live_asr.py (included in this repo) gives a pseudo-streaming experience:

• Silero VAD detects when you start and stop talking.
• While you talk, the turn is re-decoded every 0.6 s to show partial text. Words that two decodes agree on are fixed and never change.
• After 500 ms of silence, the turn is finalized with beam search.

pip install faster-whisper "av<19" sounddevice soundfile numpy
hf download TheBiggyBang/whisper-small-tunisian-derja-ct2 --local-dir whisper-tn-ct2
python whisper-tn-ct2/live_asr.py --model whisper-tn-ct2                           # microphone (GPU)
python whisper-tn-ct2/live_asr.py --model whisper-tn-ct2 --file talk.wav           # replay a file in real time
python whisper-tn-ct2/live_asr.py --model whisper-tn-ct2 --device cpu --compute-type int8 --no-partials

Measured latency (median, test audio replayed in real time):

• On CPU, use --no-partials, or a machine with more cores.
• On 10 s of pure noise, no configuration produced any text.

Files

Converted with:

ct2-transformers-converter --model TheBiggyBang/whisper-small-tunisian-derja --output_dir whisper-tn-ct2 \
  --copy_files tokenizer.json preprocessor_config.json --quantization int8_float16

Training and evaluation (summary)

Full details are on the Transformers model card.

• Base: openai/whisper-small, full fine-tuning on 2× T4 for 10.3 h (4,071 steps × batch 32). Best checkpoint at step 3,416, chosen by dev normalized WER.
• Data: the default configs of linagora/linto-dataset-audio-ar-tn (40 %) and linagora/linto-dataset-audio-ar-tn-augmented (60 %).
• Evaluation: dev and test are split by recording from all original test splits. Evaluation sentences are block-listed from training.

Limitations

• Speaker overlap is possible. The test set is recording-disjoint, not speaker-disjoint, because the data has no speaker IDs.
• Spelling. Derja has no standard spelling, and near-miss spellings are the largest error category. Report CER with WER.
• Code-switching. French/English words may come out in a different script than an annotator would use.
• Not a true streaming model. live_asr.py re-decodes the growing turn, so latency depends on your hardware.
• Inherited Whisper risks. Hallucination or repetition on hard audio. Do not use on recordings made without consent or for high-stakes decisions.

Attribution

Fine-tuned from openai/whisper-small (Apache-2.0). Training data: the LinTO DataSet Audio for Arabic Tunisian and its augmented version by LINAGORA, licensed under CC-BY-4.0. These include TunSwitch (Abdallah et al., 2023), MASC (Al-Fetyani et al., 2021) and other community and web sources listed on the dataset cards. The data was segmented, filtered and used for training; no audio or transcripts are redistributed here. live_asr.py uses Silero VAD (MIT) through faster-whisper.

Citation

@misc{linagora2024Linto-tn,
  title = {LinTO Audio and Textual Datasets to Train and Evaluate Automatic Speech Recognition in Tunisian Arabic Dialect},
  author = {Hedi Naouara and Jérôme Louradour and Jean-Pierre Lorré},
  year = {2025},
  month = {March},
  eprint={2504.02604},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  note={Good Data Workshop, AAAI 2025},
  url={arxiv.org/abs/2504.02604},
}

@misc{abdallah2023leveraging,
      title={Leveraging Data Collection and Unsupervised Learning for Code-switched Tunisian Arabic Automatic Speech Recognition},
      author={Ahmed Amine Ben Abdallah and Ata Kabboudi and Amir Kanoun and Salah Zaiem},
      year={2023},
      eprint={2309.11327},
      archivePrefix={arXiv},
      primaryClass={eess.AS}
}

@data{e1qb-jv46-21,
doi = {10.21227/e1qb-jv46},
url = {https://dx.doi.org/10.21227/e1qb-jv46},
author = {Al-Fetyani, Mohammad and Al-Barham, Muhammad and Abandah, Gheith and Alsharkawi, Adham and Dawas, Maha},
publisher = {IEEE Dataport},
title = {MASC: Massive Arabic Speech Corpus},
year = {2021} }

@misc{radford2022whisper,
  doi = {10.48550/ARXIV.2212.04356},
  url = {https://arxiv.org/abs/2212.04356},
  author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  title = {Robust Speech Recognition via Large-Scale Weak Supervision},
  publisher = {arXiv},
  year = {2022},
  copyright = {arXiv.org perpetual, non-exclusive license}
}