A 66 MB LoRA adapter that adapts CohereLabs/cohere-transcribe-arabic-07-2026 to Omani conversational Arabic. Its training labels use colloquial as-spoken spelling, keep English words in Latin script, and include Omani place names; these properties are not evaluated separately.
The adapter was trained on about 2 hours of private audio from 15 speakers. On a blind test from the same speakers it reduces word error rate (WER) by 10.9 points (29% relative) compared with the base model. On two public test sets of Gulf speakers not in the training data, it reduces WER by 1.9 points (5%) and 3.1 points (8%). On formal Modern Standard Arabic (MSA) it increases WER by 2.7 points (51%). Performance on Omani speakers outside the training group has not been measured.
| evaluation set | what it tests | normalizer | base WER | adapter WER | change |
|---|---|---|---|---|---|
| Omani blind test (22 files) | In-domain, held out, scored once | metric v2 | 37.0% | 26.1% | -10.9 pts (-29%) |
| Omani development set (64 files) | In-domain, used during development (optimistic) | metric v2 | 32.9% | 18.3% | -14.6 pts (-44%) |
| Casablanca UAE (813 utterances) | Gulf speakers not in the training data | leaderboard | 34.99% | 33.08% | -1.9 pts (-5%) |
| SADA 2022 Khaliji (1150 utterances) | Gulf speakers not in the training data | leaderboard | 36.51% | 33.43% | -3.1 pts (-8%) |
| FLEURS ar_eg (428 utterances) | Formal MSA, out of scope | leaderboard | 5.23% | 7.89% | +2.7 pts (+51%) |
WER with 95% bootstrap confidence intervals. Metric v2 for the private Omani sets (files), leaderboard normalizer for the public sets (utterances).
Suitable for: transcription of conversational Omani Arabic similar to the training data: short recordings from phone microphones, including speech that mixes Arabic and English. On Gulf broadcast speech the gains are smaller.
Not suitable for: formal MSA such as news, lectures or read text, where
the base model is more accurate; applications that need English words
rendered in Arabic script, or numbers as digits without the display pass in
postproc/; and uses that depend on accuracy for speakers or domains not
covered by the evaluations below.
All base-versus-adapter comparisons use the same decoding for both models
(greedy, max_new_tokens=432, no_repeat_ngram_size=6). Scoring is
described under Evaluation methodology.
Intervals are 95% bootstrap confidence intervals.
Two private sets of files from the training speakers, never used for training.
Recorded and labeled after the released checkpoint was trained. It was not used for checkpoint or version selection and was scored once. It is the most reliable in-domain estimate.
| condition | WER (metric v2) | 95% CI |
|---|---|---|
| base model | 37.0% | [30.8%, 42.3%] |
| base + adapter | 26.1% | [22.7%, 29.7%] |
The adapter has a lower WER on 16 of the 22 files, a higher WER on 5, and the same WER on 1 (two-sided sign test, p = 0.027). Per-file scores are private, so no paired interval is reported for this set.
Consulted repeatedly during development, so its results are optimistic.
| condition | WER (metric v2) | 95% CI | CER (metric v2) | WER (strict) |
|---|---|---|---|---|
| base model | 32.9% | [30.1%, 35.7%] | 20.7% | 53.3% |
| base + adapter | 18.3% | [16.5%, 20.1%] | 8.0% | 27.9% |
The optional glossary in postproc/ does not change these results: none of
its rules applies to the adapter's output on this set.
The UAE portion of Casablanca and the Khaliji speakers of SADA 2022. Neither contains speakers from the training data.
| set | N | condition | WER (leaderboard) | WER (metric v2) | CER (leaderboard) | CER (metric v2) |
|---|---|---|---|---|---|---|
| Casablanca UAE test | 813 | base model | 34.99% [33.40, 36.64] | 40.62% [39.06, 42.16] | 12.63% | 12.59% |
| Casablanca UAE test | 813 | base + adapter | 33.08% [31.64, 34.57] | 38.72% [37.34, 40.21] | 11.31% | 11.33% |
| SADA 2022 test, Khaliji speakers | 1150 | base model | 36.51% [34.74, 38.27] | 36.67% [34.87, 38.42] | 17.25% | 17.38% |
| SADA 2022 test, Khaliji speakers | 1150 | base + adapter | 33.43% [31.83, 35.16] | 34.00% [32.44, 35.72] | 14.44% | 14.92% |
The adapter lowers CER by 1.32 points (10%) on Casablanca UAE and by 2.81 points (16%) on SADA 2022 Khaliji.
The full FLEURS ar_eg test set (428 utterances):
| set | N | condition | WER (leaderboard) | WER (metric v2) | CER (leaderboard) | CER (metric v2) |
|---|---|---|---|---|---|---|
| FLEURS ar_eg test | 428 | base model | 5.23% [4.25, 6.50] | 5.05% [4.05, 6.38] | 2.26% | 2.24% |
| FLEURS ar_eg test | 428 | base + adapter | 7.89% [6.69, 9.29] | 7.32% [6.20, 8.77] | 4.87% | 4.67% |
Part of this gap reflects output conventions. The adapter writes English words in Latin script and numbers as Arabic words, as in its training labels. FLEURS references use Arabic script and, for numbers, digits:
| transcripts containing... | base model | base + adapter | reference transcripts |
|---|---|---|---|
| Latin-script words | 1 | 79 | 13 |
| digits | 59 | 30 | 55 |
Counts out of 428 utterances, for the references, the base model and the adapter.
On the 348 utterances where neither the reference nor either hypothesis contains a Latin-script word, the adapter's WER is 1.06 points higher than the base model's (paired 95% CI [0.44, 1.71], leaderboard normalizer) and 0.72 points higher under metric v2 (CI [0.20, 1.27]). Script choice therefore accounts for most of the gap, and the remainder is a loss of accuracy on formal speech.
Both models transcribe the same utterances, so we compare them with a paired bootstrap that resamples utterances jointly for both. Under both normalizers, the 95% interval for the WER difference lies below zero on the Casablanca and SADA sets and above zero on FLEURS:
| set | N | WER change, leaderboard [paired 95% CI] | WER change, metric v2 [paired 95% CI] | utterances: adapter better / base better / tie |
|---|---|---|---|---|
| Casablanca UAE test | 813 | -1.90 [-3.19, -0.79] | -1.90 [-3.07, -0.90] | 265 / 204 / 344 |
| SADA 2022 test, Khaliji speakers | 1150 | -3.08 [-4.51, -1.77] | -2.68 [-4.08, -1.43] | 365 / 269 / 516 |
| SADA 2022 test, all dialects | 6189 | -2.64 [-3.19, -2.06] | -1.16 [-1.70, -0.59] | 2146 / 1693 / 2350 |
| FLEURS ar_eg test | 428 | +2.65 [+1.97, +3.39] | +2.27 [+1.65, +2.89] | 39 / 123 / 266 |
The last column counts utterances on which the adapter makes fewer word errors than the base model, more, or the same number, under the leaderboard normalizer.
Omani blind test: files, metric v2. Public sets: utterances, leaderboard normalizer.
Paired 95% bootstrap intervals, under both normalizers.
Part of the gain on the public sets also reflects conventions. Metric v2 ignores number format and hesitation fillers, and under it the gains are smaller: -1.16 points against -2.64 on SADA 2022 all dialects, and -2.68 against -3.08 on SADA 2022 Khaliji.
To check the evaluation pipeline, we scored the base model on the full SADA 2022 test list used by the Open Universal Arabic ASR Leaderboard (6189 segments, all dialects) with the leaderboard normalizer and bracketed markers counted as words, as the leaderboard does:
| condition | decoding | normalizer | WER |
|---|---|---|---|
| base model | recommended (max_new_tokens=432, no_repeat_ngram_size=6) | leaderboard, no marker handling | 35.72% [34.96, 36.53] |
| base model | base card default (max_new_tokens=256, no n-gram blocking) | leaderboard, no marker handling | 36.53% [35.45, 37.71] |
| base model | base card default | as published by CohereLabs | 37.47% |
The number published on the base model card lies within the confidence interval of our run with the same decoding. Scored the same way with the recommended decoding, the adapter's WER is 2.56 points lower than the base model's (paired difference).
The adapter sometimes emits bracketed non-speech markers (see Output conventions). The number of markers emitted by each model is shown below; the number of hypotheses that contain at least one is in parentheses.
| set | N | base model: markers (hypotheses) | base + adapter: markers (hypotheses) |
|---|---|---|---|
| SADA 2022 test, all dialects | 6189 | 7 (5) | 179 (92) |
| Casablanca UAE test | 813 | 0 (0) | 2 (2) |
| FLEURS ar_eg test | 428 | 0 (0) | 0 (0) |
The public-set tables above remove markers from hypotheses before scoring. Counting them as words instead changes the adapter's SADA result by less than 0.1 points:
| scoring of bracketed markers | adapter WER on SADA (all dialects) |
|---|---|
| bracketed markers stripped from hypotheses (card convention) | 33.07% [32.26, 33.85] |
| markers counted as words, as the leaderboard would score | 33.15% [32.34, 33.90] |
Scoring. WER and CER are computed after normalizing reference and hypothesis in the same way. We compare numbers only within one scoring:
normalize_arabic_text from the
Open Universal Arabic ASR Leaderboard
at commit 8fce8595b490. It removes punctuation and diacritics, unifies
hamza forms, and attaches a standalone و to the following word. CER
includes spaces, as in NeMo's word_error_rate. This is the headline
scoring for public sets. In our tables, bracketed markers are removed from
hypotheses first.metrics/normalize_ar.py in this repository, with 21 unit
tests. It removes diacritics and punctuation; unifies hamza, taa marbuta
and alef maqsura; treats number words and digits as equal (سبعة عشر and
17); removes hesitation fillers; and maps 23K to 23000 and cheque
to check. It measures word recognition rather than spelling conventions.
CER excludes spaces. This is the scoring for the private Omani sets.The normalizers can differ by several points on the same transcripts. On
Casablanca UAE, the base model scores 34.99% with the leaderboard
normalizer and 40.62% with metric v2. The difference comes from
the conjunction و, which is a separate word in 4.7% of
Casablanca reference words (against 0.5% in SADA); only
the leaderboard normalizer attaches it to the next word. Adding that one
rule to metric v2 gives 34.63%, within 0.4
points of the leaderboard score. On SADA 2022 Khaliji the two normalizers
differ by only 0.16 points.
Identical transcripts; the axis is zoomed to the range of the scores.
Confidence intervals. 95% percentile bootstrap with 1000 resamples, over files for the private sets and over utterances for the public sets. Paired intervals resample utterances jointly for both models.
Audio. Public-set audio is resampled to 16 kHz mono. SADA segments are cut from whole-show recordings using the dataset's segment boundaries.
The adapter follows the conventions of its training labels:
عطني rather than أعطني), with fillers
kept.أحد عشر
rather than حدعش).[غير واضح] for unintelligible
speech and [ضحك] for laughter. On public broadcast audio the adapter
mostly emits [ضحك] and variants such as [ضحكة] and [ضحى], and
occasionally speaker labels such as [SpeakerA].postproc/ provides optional post-processing: a glossary that standardizes
spellings, a formatter that converts number words to digits for display, and
markers.py, which removes or maps bracketed markers. Reported metrics are
computed on raw output, except that bracketed markers are removed as
markers.py does. The glossary has no effect on the development set. The
digit formatter does not affect metric v2 but would change scores under the
leaderboard normalizer.
import soundfile as sf
import torch
from peft import PeftModel
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
BASE = "CohereLabs/cohere-transcribe-arabic-07-2026" # accept its terms on the Hub first
REV = "c3e911b42149bf7a1e53d5cef9878aee87515a23"
proc = AutoProcessor.from_pretrained(BASE, revision=REV)
dev = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if dev == "cuda" else torch.float32
model = AutoModelForSpeechSeq2Seq.from_pretrained(BASE, revision=REV, dtype=dtype).to(dev)
model = PeftModel.from_pretrained(model, "Rihal-om/omani-transcribe-lora")
model.eval()
audio, sr = sf.read("audio_16k_mono.wav", dtype="float32") # 16 kHz mono
inp = proc(audio=audio, sampling_rate=sr, language="ar", return_tensors="pt")
inp = {k: (v.to(dev) if hasattr(v, "to") else v) for k, v in inp.items()}
inp = {k: (v.to(dtype) if hasattr(v, "dtype") and v.is_floating_point() else v) for k, v in inp.items()}
ci = inp.pop("audio_chunk_index", None)
with torch.no_grad():
# no_repeat_ngram_size=6 prevents a greedy-decoding failure in which the
# model repeats a dash or filler and drops the rest of the chunk.
# max_new_tokens=432 stays within the decoder's 1024 positions.
out = model.generate(**inp, max_new_tokens=432, no_repeat_ngram_size=6)
text = (proc.decode(out, audio_chunk_index=ci, language="ar", skip_special_tokens=True)
if ci is not None else proc.decode(out[0], skip_special_tokens=True))
print(" ".join(text) if isinstance(text, list) else text)
The processor splits audio longer than 35 s into 35 s windows with 5 s of
overlap. Keep the default <|noitn|> prompt mode, which the adapter was
trained with. proof/clean_env_test.py runs this example in a fresh
environment, and sample/ contains a consented demo clip with its reference
transcript.
About 2 hours of labeled Omani Arabic audio from 15 speakers in one family and workplace group, all of whom consented to the use of their recordings for training this model. The two most frequent speakers account for about 25% and 21% of the audio. All recordings are short conversational clips from phone microphones, stored as Opus. Labels are human corrections of a commercial ASR draft, so they may retain some of that system's spelling choices. The data is private and will not be released.
target_modules="all-linear"), no bias. The base model is
frozen in fp16; LoRA weights are trained in fp32 and saved in fp16.c3e911b42149bf7a1e53d5cef9878aee87515a23.ae8714661ea244b99e80c1959da7f60abee5595451d6431ac0cbd6e674af6941. The
same hash is recorded in every adapter run in eval/hypotheses/.results/results.json holds every reported value. This README, its tables,
its numbers and its figures are generated from that file, and
results/check_readme.py verifies that they agree.eval/ contains the public-set evaluation: code, decoding configs,
per-utterance hypotheses for both models, run metadata and scores.
Reference transcripts are not included because the benchmark licenses do
not permit redistribution (Casablanca is CC-BY-NC-ND-4.0).
eval/src/attach_refs.py downloads them from the Hub and verifies each
rebuilt file against the hash of the original; eval/src/score.py and
eval/src/paired.py then reproduce every public-set number above. See
eval/README.md.google/fleurs), Casablanca UAE test
(UBC-NLP/Casablanca, config UAE) and the SADA 2022 test list of the
Open Universal Arabic ASR Leaderboard (m6011/sada2022).postproc/markers.py.no_repeat_ngram_size, greedy decoding can
enter a repetition loop and drop the rest of a 35 s chunk, for both the
base model and the adapter. All reported results use
no_repeat_ngram_size=6.Apache 2.0. The adapter is a derivative of CohereLabs/cohere-transcribe-arabic-07-2026, which is itself fine-tuned from CohereLabs/cohere-transcribe-03-2026; both are released by Cohere Labs under Apache 2.0. The base model weights are not included in this repository and must be downloaded from their own repositories.
@misc{rihal2026omanilora,
title = {Omani Arabic LoRA for Cohere Transcribe Arabic},
author = {Rihal},
year = {2026},
url = {https://huggingface.co/Rihal-om/omani-transcribe-lora}
}
Released and maintained by Rihal.