ivrit.ai Whisper large-v3-turbo, GGUF for transcribe.cpp

A Hebrew speech-to-text model: ivrit.ai's fine-tune of OpenAI's Whisper large-v3-turbo, converted to GGUF so it runs in transcribe.cpp on CPU, Vulkan, Metal, and CUDA. It is the format the Hex dictation app loads.

This repository changes only the file format. The model, its training, and its data are ivrit.ai's work.

On the full FLEURS Hebrew test split it makes 18.46% word errors, against 29.51% for stock Whisper large-v3-turbo measured the same way: about 37% fewer errors, at the same speed and size.

Credits

• ivrit.ai trained the model (ivrit-ai/whisper-large-v3-turbo, training code) on their open Hebrew datasets:

  • crowd-transcribe-v5: about 300 hours of public audio, transcribed segment by segment by volunteers;
  • crowd-recital-whisper-training: about 50 hours of volunteers reading Wikipedia passages;
  • knesset-plenums-whisper-training: about 4,700 hours of Knesset plenary sessions.

  Thanks to everyone who transcribed and recorded for those datasets. Please support ivrit.ai's work and see their Hebrew transcription leaderboard.

• OpenAI released the base model, Whisper large-v3-turbo.

• The transcribe.cpp authors wrote the converter, quantizer, runtime, and WER harness used here.

• Google published FLEURS (CC BY 4.0), the evaluation set below.

Use it for Hebrew only

ivrit.ai notes that the fine-tune degraded Whisper's language detection and translation. Always set the language to Hebrew (he), never auto-detect, and don't ask it to translate. It is meant for mostly-Hebrew speech.

Files

Use the Q8_0 file. The F32 file is the unquantized conversion, for anyone who wants to quantize differently with transcribe-quantize.

Usage

With transcribe.cpp's CLI and a 16 kHz mono WAV:

transcribe-cli -m ivrit-whisper-large-v3-turbo-Q8_0.gguf -l he audio.wav

Accuracy

The full FLEURS Hebrew (he_il) test split, 792 utterances, through transcribe.cpp's WER harness (scripts/wer/run.py and score.py; language he, no timestamps, greedy decoding, batch size 1) at transcribe.cpp 7045d1c, Q8_0 on Vulkan (NVIDIA RTX 3090):

The stock figure reproduces transcribe.cpp's published 29.71% for the same file and split, so the two rows are comparable. FLEURS is read speech; ivrit.ai's leaderboard covers other kinds of Hebrew audio.

How it was made

From ivrit-ai/whisper-large-v3-turbo at f33172a8c3c6efbc040a7200e00835257cac0447, with transcribe.cpp at 7045d1c0fe7cee37747b2792977421d435e75fca plus one line in VARIANT_DISPLAY_NAMES in scripts/convert-whisper.py, so the file carries this model's name rather than OpenAI's:

"ivrit-whisper-large-v3-turbo": "ivrit.ai Whisper Large v3 Turbo",

uv sync --project scripts/envs/whisper
uv run --project scripts/envs/whisper python scripts/convert-whisper.py \
  ivrit-ai/whisper-large-v3-turbo ivrit-whisper-large-v3-turbo-F32.gguf \
  --revision f33172a8c3c6efbc040a7200e00835257cac0447 \
  --variant ivrit-whisper-large-v3-turbo
transcribe-quantize ivrit-whisper-large-v3-turbo-F32.gguf \
  ivrit-whisper-large-v3-turbo-Q8_0.gguf --quant Q8_0

The source checkpoint is F32 and ships no mel filterbank; the converter computes it with transformers.audio_utils. The GGUF's general.author and general.organization are ivrit-ai, general.license is apache-2.0, and general.repo_url points at ivrit.ai's repository. Quantizing again from the F32 file reproduces the Q8_0 byte for byte.

License

Apache-2.0, the license of ivrit.ai's model. The Whisper large-v3-turbo base model is MIT-licensed by OpenAI.