ufakzeka-kulak

ufakzeka-kulak ("kulak" is Turkish for ear) is a small Turkish speech-to-text model by ufak AI, trained from scratch and released under Apache-2.0. It runs offline on a CPU, and the same weights ship for NeMo, transformers, sherpa-onnx, parakeet.cpp (GGUF) and parakeet-mlx (MLX).

Read this before you try it

The model is meant for transcribing Turkish speech offline on a laptop, a server or in a browser page, where a licence that allows commercial use and a known list of training sources matter. Identifying, tracking or profiling the people who speak is out of scope, and so is acting on a transcript about a person without someone checking it.

Limits:

• Turkish only. Other languages, and Turkish mixed with them, were not measured.
• One utterance at a time. The released model is not a streaming model. Cut long recordings into utterances with voice activity detection and keep a few hundred milliseconds of audio around each one, as the demo does; a segment cut tight to the speech can come out without capitals or punctuation.
• The model does not say who is speaking. Word timestamps were not evaluated.
• Numbers come out as words. An application that needs digits must convert them.
• Commas are often missed, with a comma recall of 0.359 on FLEURS Turkish test and 0.612 on Common Voice 27.0 Turkish test, against 0.839 and 0.939 for full stops.
• Conversational speech is much harder. On Real-TurnTurk, a conversational test set with 6 speakers, normalised WER is 0.3807, against 0.0921 on FLEURS Turkish test; with so few speakers its interval is wide.
• Most of the training text of the YODAS recordings (4,326.6 of 4,696.2 hours) was written by one speech recogniser and checked by another, wherever no uploader subtitle agreed with that transcript (see Training data). Errors on which the two agreed remain in the training text.
• Accuracy by speaker gender, age or regional accent was not measured.
• Each runtime has its own limits, listed under Known limitations.

Model summary

Output. The model writes Turkish with capital letters, full stops, commas, question marks, apostrophes and hyphens inside words. Other punctuation in the training text was mapped to these (an exclamation mark to a full stop, a semicolon or a colon to a comma). Numbers are written as words, so the model writes "iki bin yirmi ikide", not 2022'de. Circumflex letters are written without the circumflex.

Files and how to run them

"Checked" means that the file was decoded on Common Voice dev and FLEURS Turkish dev and its normalised WER stayed within 0.0050 of NeMo's on both sets, with no failed utterance; a reduced copy (int8, q8_0, or the MLX weights run at bfloat16) was also held to its full-precision copy by the same margin. The sherpa-onnx, parakeet.cpp and parakeet-mlx checks ran under Linux x86-64; parakeet.cpp and parakeet-mlx ran on the CPU. One SHA256SUMS file at the top level lists every file of the release, LICENSE and ATTRIBUTION.md included, except this card, SHA256SUMS itself and the Hub's .gitattributes. Add "SHA256SUMS" to the allow_patterns below, then sha256sum -c --ignore-missing SHA256SUMS (GNU coreutils) in the download folder checks the files you fetched.

Everything is in this one repo. The NeMo file sits at the top level, where NeMo's from_pretrained looks for it, and NeMo downloads only that file; each other runtime has its own folder, and the examples below download only the folder they use.

The examples below read speech.wav, one utterance of mono audio at 16,000 Hz. The install lines name the versions the files were checked with.

NeMo

The checkpoint holds both the TDT decoder and the CTC head; TDT is the default.

pip install "nemo_toolkit[asr]==3.0.0"

from nemo.collections.asr.models import ASRModel

model = ASRModel.from_pretrained("ufakai/ufakzeka-kulak")
print(model.transcribe(["speech.wav"])[0].text)

model.change_decoding_strategy(decoder_type="ctc")  # the CTC head
print(model.transcribe(["speech.wav"])[0].text)

transformers

pip install transformers==5.20.1 torch soundfile==0.14.0 huggingface_hub==1.33.0

import soundfile as sf
import torch
from huggingface_hub import snapshot_download
from transformers import ParakeetForTDT, ParakeetProcessor

root = snapshot_download("ufakai/ufakzeka-kulak", allow_patterns=["transformers-tdt/*"])
folder = f"{root}/transformers-tdt"
processor = ParakeetProcessor.from_pretrained(folder)
model = ParakeetForTDT.from_pretrained(folder, dtype=torch.float32).eval()
audio, rate = sf.read("speech.wav", dtype="float32")
if audio.ndim > 1:  # a stereo file: average the channels
    audio = audio.mean(axis=1)
inputs = processor(audio=audio, sampling_rate=rate, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs)
print(processor.decode(out.sequences[0], skip_special_tokens=True))

ParakeetForCTC with the transformers-ctc folder works the same way (generate returns the token ids; decode them with processor.batch_decode). pipeline("automatic-speech-recognition", model=folder) also runs both folders; its text matched generate on 10 of 10 utterances of each development set, for both folders.

sherpa-onnx

Two settings belong to the use of these files. The input gain multiplies the samples before decoding; it is 8 for the shipped files. A gain above one is a workaround for a difference between the sherpa-onnx front end and NeMo's (see Known limitations), and it holds at normal recording levels, not on quiet audio. For the CTC file, the caller pad appends zeros after the gain so that the runtime keeps the last encoder frame, which often carries the final punctuation mark. The pad is defined for audio at 16,000 Hz, the model's input rate; resample other audio first.

pip install sherpa-onnx==1.13.8 numpy soundfile==0.14.0 huggingface_hub==1.33.0

import numpy as np
import sherpa_onnx
import soundfile as sf
from huggingface_hub import snapshot_download

GAIN = 8  # the input gain of the shipped files

def caller_pad(samples, hop=160, multiple=8):
    """Zeros up to the next multiple of 8 feature frames (CTC file only, 16 kHz audio)."""
    frames = (len(samples) + hop // 2) // hop
    target = -(-frames // multiple) * multiple
    pad = max(0, hop * target - hop // 2 - len(samples))
    return np.concatenate([samples, np.zeros(pad, dtype=np.float32)])

root = snapshot_download("ufakai/ufakzeka-kulak", allow_patterns=["sherpa-onnx-tdt/*", "sherpa-onnx-ctc/*"])
audio, rate = sf.read("speech.wav", dtype="float32")
if audio.ndim > 1:  # a stereo file: average the channels
    audio = audio.mean(axis=1)
assert rate == 16000, "resample to 16 kHz first: the model and the caller pad expect it"
samples = (audio * np.float32(GAIN)).astype(np.float32)

tdt = sherpa_onnx.OfflineRecognizer.from_transducer(
    encoder=f"{root}/sherpa-onnx-tdt/encoder.int8.onnx",
    decoder=f"{root}/sherpa-onnx-tdt/decoder.int8.onnx",
    joiner=f"{root}/sherpa-onnx-tdt/joiner.int8.onnx",
    tokens=f"{root}/sherpa-onnx-tdt/tokens.txt",
    sample_rate=16000, feature_dim=80,
    decoding_method="greedy_search", model_type="nemo_transducer",
)
s = tdt.create_stream()
s.accept_waveform(rate, samples)
tdt.decode_stream(s)
print(s.result.text)

ctc = sherpa_onnx.OfflineRecognizer.from_nemo_ctc(
    model=f"{root}/sherpa-onnx-ctc/model.int8.onnx",
    tokens=f"{root}/sherpa-onnx-ctc/tokens.txt",
    sample_rate=16000, feature_dim=80, decoding_method="greedy_search",
)
s = ctc.create_stream()
s.accept_waveform(rate, caller_pad(samples))
ctc.decode_stream(s)
print(s.result.text)

The files without .int8 are the float32 copies. The browser demo runs the int8 TDT files at gain 8 in the sherpa-onnx WebAssembly build; the Android and iOS builds were not tried. In any build, apply the gain and, for CTC, the pad before decoding.

GGUF with parakeet.cpp

One GGUF file holds both the TDT decoder and the CTC head. Download it, build the command-line tool from the tag named above and choose the decoder with --decoder. The build switches off the parts of parakeet.cpp that need other submodules, as the checked build did.

pip install huggingface_hub==1.33.0
hf download ufakai/ufakzeka-kulak gguf/ufakzeka-kulak-f32.gguf --local-dir .
git clone --branch v0.6.1 https://github.com/mudler/parakeet.cpp
cd parakeet.cpp
git submodule update --init third_party/ggml
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DPARAKEET_BUILD_CLI=ON -DPARAKEET_BUILD_SERVER=OFF \
    -DPARAKEET_BUILD_TESTS=OFF -DPARAKEET_WITH_CED=OFF -DPARAKEET_WITH_VOICEDETECT=OFF
cmake --build build --target parakeet-cli
./build/examples/cli/parakeet-cli transcribe --model ../gguf/ufakzeka-kulak-f32.gguf --input ../speech.wav
./build/examples/cli/parakeet-cli transcribe --model ../gguf/ufakzeka-kulak-f32.gguf --input ../speech.wav --decoder ctc

The q8_0 file, gguf/ufakzeka-kulak-q8_0.gguf, is used the same way.

MLX with parakeet-mlx

parakeet-mlx computes its input features differently from NeMo. With its own transcribe at bfloat16, Common Voice dev WER is 0.0915 against NeMo's 0.0830, outside the check's margin. This folder therefore ships its own feature function, mlx_frontend.py, which computes NeMo's features with numpy; compute the features with it and pass them to the model's generate. The weights are stored in float32. parakeet-mlx runs them at bfloat16 by default, and bfloat16 is the checked setting. Two full runs at bfloat16 gave identical token ids on every development utterance and both passed the check.

pip install parakeet-mlx==0.5.3 mlx==0.32.4 huggingface_hub==1.33.0

import json
import sys

import mlx.core as mx
import numpy as np
from huggingface_hub import snapshot_download
from parakeet_mlx import from_pretrained
from parakeet_mlx.audio import load_audio

folder = snapshot_download("ufakai/ufakzeka-kulak", allow_patterns=["mlx/*"]) + "/mlx"
sys.path.insert(0, folder)
from mlx_frontend import features

model = from_pretrained(folder)  # runs at bfloat16, the checked setting
with open(f"{folder}/config.json", encoding="utf-8") as f:
    preprocessor = json.load(f)["preprocessor"]
samples = np.array(load_audio("speech.wav", preprocessor["sample_rate"]))
mel = mx.array(features(samples, preprocessor)[None])
print(model.generate(mel)[0].text)

parakeet-mlx decodes the TDT head only. load_audio needs ffmpeg.

Training data

Every training input has a licence that allows commercial use. Hours count each recording once; the speed-perturbed copies made during training are not counted again.

The YODAS recordings are YouTube audio collected by YODAS and YODAS v3, whose authors say they took only videos published under a Creative Commons licence; they release the audio under CC BY 3.0. On YouTube the one Creative Commons licence an uploader can choose is the Creative Commons Attribution licence (CC BY). The three parts are the Turkish yodas0.10 subset of ESPnet's OWSM v4 data, the yodas0.00 recordings not in it, and Turkish YODAS v3 audio cut along the spans of Alania Turkish Speech Style Captions (CC BY 4.0). The Alania dataset supplied only those span boundaries; its transcript column was never read. The training text is chosen per utterance. Where the utterance has an uploader subtitle whose WER against the Whisper large-v3-turbo transcript is at most 0.2000, the subtitle is the training text. Otherwise (no subtitle, or one that disagrees) it is the Whisper large-v3-turbo transcript, kept only where Omnilingual ASR CTC 7B v2 reads the same audio within a character error rate of 0.1000. YouTube's automatic captions are never training text.

ISSAI TSC. ISSAI publishes the corpus under CC BY 4.0 (LICENSE.md of https://github.com/IS2AI/TurkicASR); the Hugging Face copy is tagged MIT. CC BY 4.0 is the licence followed here.

Models used as data tools. Whisper large-v3-turbo (MIT) wrote the transcripts. Omnilingual ASR CTC 7B v2 (Apache-2.0) checked them and chose the spoken reading of number forms. Both were chosen on the ISSAI TSC development split, which was never trained on, against Whisper large-v3 (MIT), Omnilingual ASR CTC 3B v2 (Apache-2.0) and Qwen3-ASR-1.7B (Apache-2.0). The makers of these models do not fully disclose their training data. No output of these models other than the transcripts and readings above is training text.

Augmentation. Noise from the MUSAN noise subset (CC BY 4.0; only files that MUSAN's licence files mark as public domain or CC BY) was mixed into training audio at random, reverberation was added at random with simulated room responses from the OpenSLR room impulse response database (Apache-2.0), the input level was varied at random, and noise clips with an empty transcript were added as examples of non-speech. The YODAS2 recording lists (CC BY 3.0) were read as metadata only.

Decontamination. One exclusion set holds the normalised references of every development and test set used here. It was matched against every training transcript and every tokenizer line, by exact match and by shared word n-grams, and audio fingerprints matched the training audio against the evaluation audio. The matches removed 432 training utterances, 206 of them ISSAI TSC clips matched by their audio. FLEURS (any split), the Common Voice development and test splits, Real-TurnTurk and the 2,511 MediaSpeech clips reported here were never trained on.

Evaluation

How the numbers were produced

The recipe, the checkpoint, the decoder settings and the export files with their settings were chosen on the development sets, Common Voice 27.0 Turkish dev and FLEURS Turkish dev. The labelling choices (the transcribing model, the checking model and its character error rate threshold) were made on the ISSAI TSC development split. The test sets were scored once, for one model, after those choices; the first release-gate attempt passed. NeMo decoding is greedy, float32, TDT with at most 10 tokens per frame. Normalised WER applies our Turkish normaliser to reference and hypothesis (bracketed spans removed, digits written as words, Turkish lower-casing, in-word apostrophes and punctuation removed, circumflex folded, filler words such as ı, ee, hmm and aa dropped); raw-cased WER compares the texts as written.

The intervals in square brackets come from resampling the test sets (a blockwise percentile bootstrap with 2,000 resamples, seed 0, 95 percent intervals), in blocks of speakers for Common Voice and Real-TurnTurk, of sentences for FLEURS (FLEURS publishes no speaker ids, so its interval ignores speaker correlation) and of clips for MediaSpeech. A difference between two systems is paired, with both resampled by the same draws. The model was trained once, with one seed, so the intervals do not cover training variance. Whisper base, Whisper small and Whisper large-v3 (MIT; their maker does not fully disclose their training data) were scored by us with the same normaliser, with greedy decoding and, for base and small, also with the Whisper package's decoding settings (beam search with 5 beams and temperature fallback) as transformers implements them. Each comparison uses the better of the two on that set, which on every set reported here is the Whisper package's settings.

Release gate

The release required the model to beat Whisper base on FLEURS Turkish test and on Common Voice 27.0 Turkish test, each by a paired interval that lies wholly below zero. The model passed on both test sets.

On FLEURS Turkish test and Common Voice 27.0 Turkish test, Whisper large-v3 (greedy) scores 0.0604 and 0.0767, lower than ufakzeka-kulak; its difference was not tested.

Normalised CER is 0.0335 [0.0258, 0.0426] on FLEURS Turkish test and 0.0242 [0.0228, 0.0258] on Common Voice 27.0 Turkish test. On the development sets normalised WER is 0.0830 on Common Voice dev and 0.1071 on FLEURS dev.

Punctuation and casing

The model writes capitals and punctuation, so the text as written is scored too, on the three test sets, against references whose punctuation is mapped to the model's marks. PC-WER counts marks and capitals; PuncER and CaseER are the parts of it that come from marks and from capitals (case and punctuation rates; the count of reference words with a capital is our reading of a term that paper does not define); PER is the punctuation error rate of a second paper, computed as NeMo's own implementation does.

On conversational speech the model writes few marks and often misplaces capitals. On Real-TurnTurk its full-stop recall is 0.171 and its comma recall 0.031, and its capitals have a precision of 0.365 and a recall of 0.272. The intervals, the per-mark counts, precision and recall and an apostrophe rate are in the result file. The WER without marks or casing removes only the marks and the capitals, with the numbers in the references written as the model writes them, so it differs from the normalised WER of the test results. The last column describes the references, not the model, since a set whose references seldom end in a mark penalises a model that writes one.

MediaSpeech Turkish

MediaSpeech (NTR Lab, CC BY 4.0) is media speech. Its makers say that part of its utterances come from Mozilla Common Voice, so every clip was checked against the Common Voice 27.0 sentences, the Common Voice development and test audio and our training audio before scoring. Of 2,513 clips, 2 matched training audio and were removed, and no clip matched Common Voice; 2,511 utterances, 10.0 hours, are reported. The references are not edited before normalisation. They join compound numbers into one word, so a CER with spaces removed is given as well.

The dataset has no speaker or video ids, so the interval treats every clip as independent and may be too narrow. Whisper's training data is not fully disclosed, so this set may not be unseen for Whisper.

Low-bitrate, telephone and level copies

Three test sets, FLEURS Turkish test, Common Voice 27.0 Turkish test and Real-TurnTurk, were also scored on stored copies of their audio; MediaSpeech has no copies. The copies are the audio compressed with the Opus audio codec at 6,000 and 12,000 bit/s, the telephone band at 8,000 Hz with A-law coding, and the original audio with its level changed by -20 and -30 dB. The table gives the WER change against the same model on the original audio (copy minus original; the paired intervals are in the result file).

Per shipped file

The NeMo numbers above are the reference. The int8 sherpa-onnx files the examples load were scored at gain 8 on the same test sets and copies, paired on the same utterances with NeMo's greedy TDT decoding (file minus NeMo); the float32 files and the other copies are in the result file. For the CTC file that difference also holds the gap between the two heads; against NeMo's own CTC decoding the CTC file differs by +0.0032 on Common Voice dev.

The other files were checked on the development sets only, against NeMo's own token ids.

Each runtime is compared with the NeMo decoding that follows its own rule at the per-frame token cap.

Speed

The shipped sherpa-onnx files were timed with sherpa-onnx 1.13.8 on 60 development utterances from Common Voice dev and FLEURS Turkish dev, the same number in each of three length buckets, short (two to five seconds), medium (five to fifteen seconds) and long (fifteen to thirty seconds). Each row ran in a fresh process. The recognizer was built from the files (cold load), the sample was decoded once without timing and then five times with timing, each utterance in its own stream, at input gain 8 and, for the CTC file, with the caller pad. The real-time factor (RTF) is decode time divided by audio time over a bucket, the median of the five timed passes; below one is faster than real time. A decode cell gives the median and the ninetieth percentile of the per-utterance decode time in seconds. With voice activity detection, the text of a segment appears after the detector's minimum silence (0.5 s in the sherpa-onnx WebAssembly default) plus that decode time. Peak memory is the peak resident set size of the measuring process. The result files hold every row with its smallest and largest pass and its provenance, and each Python row with the load average just before and after it.

Apple M1, sherpa-onnx Python:

Chrome on the same Apple M1 (Chrome 155.0.8059.84, headless), the demo page's sherpa-onnx WebAssembly build with its int8 TDT files, single-threaded. Here the memory column is the JavaScript heap after the timed passes (performance.memory.usedJSHeapSize), which leaves out the WebAssembly memory.

A 2-vCPU x86-64 Linux cloud server (Intel(R) Xeon(R) CPU @ 2.20GHz), sherpa-onnx Python, at one and at two threads. The server is a shared virtual machine whose CPU model was recorded, not chosen. The load averages and steal around each row are in the result file. On this server every two-thread row had a higher median real-time factor than the same file at one thread.

Known limitations

• sherpa-onnx front end. The sherpa-onnx feature front end differs from NeMo's in six options. A comparison on an earlier model of the same architecture, with its own data and tokenizer, found that of the differing options only the small constant added before the logarithm, which weighs most on low-energy frames, closed the gap; the others moved WER slightly. The input gain the sherpa-onnx section names is a workaround that holds at normal levels; the level copies above show what is left on quiet audio. The upstream change is pull request 3999, merged on 21 October 2026 and released in sherpa-onnx 2.0.0, which was not measured here; with 1.13.8, use the gain.
• sherpa-onnx CTC. The sherpa-onnx NeMo CTC runtime sets the output length to the feature frame count divided by the time reduction, rounded down, instead of the encoder's own length, so the last encoder frame is dropped whenever the frame count is not a multiple of it. That frame can carry the final punctuation mark. Normalised WER does not see the loss; raw-cased WER and the punctuation rates do. The caller pad in the sherpa-onnx section restores the frame. On Common Voice dev, of 11,864 hypotheses, 1,788 end in a mark without the pad, 11,637 with it and 11,644 with NeMo's own CTC decoding. The TDT files are not affected.
• Quiet audio. The model was trained with random input levels; the level copies in the Evaluation section measure how much accuracy it still loses on quiet recordings, for NeMo and for the two int8 sherpa-onnx files.
• MediaSpeech reference style. Its references join compound numbers into one word, so part of its WER comes from the reference style; the CER without spaces does not count those joins. Its interval may be too narrow (see MediaSpeech Turkish).
• Apple silicon. The parakeet-mlx files were checked on Linux CPU only. They were not run on Apple silicon or on mlx's Metal backend; the bfloat16 behaviour there may differ.
• transformers decodes TDT without NeMo's per-frame token cap; on the development sets this changed no utterance (11,864 of 11,864 Common Voice dev and 338 of 338 FLEURS dev utterances give NeMo's token ids).

Attribution

ATTRIBUTION.md in this repo lists every source with its licence. In short:

Mozilla Common Voice contributors (CC0 1.0); Antalia (Patientdesk.ai), antalia-voice-corpus (CC BY 4.0); the Institute of Smart Systems and Artificial Intelligence (ISSAI), Nazarbayev University, Turkish Speech Corpus (CC BY 4.0); YODAS, the OWSM v4 data and YODAS v3 by ESPnet (CC BY 3.0); Alania Turkish Speech Style Captions by PatientDesk AI (https://huggingface.co/datasets/cloud0day3/alania-speech-captions-tr), licensed under CC BY 4.0, © 2026 PatientDesk AI; MUSAN (CC BY 4.0) and the OpenSLR simulated room responses (Apache-2.0) for augmentation. The evaluation sets are FLEURS (Google, CC BY 4.0), Common Voice 27.0 Turkish dev and test (CC0 1.0), Real-TurnTurk (Apache-2.0) and MediaSpeech (NTR Lab, CC BY 4.0).

The antalia-voice-corpus card asks that its audio, and anything trained on it, not be used to impersonate the speaker or any real person or to imply someone said something they did not, nor for fraud, scams, social engineering or political robocalls, and that synthetic speech made from it be disclosed. ufakzeka-kulak produces no speech; users of this model are asked to honour these requests.

Citation

The paper describes the method and the results; the second entry cites the released model.

@misc{teke2026ufakzekakulak,
  title         = {ufakzeka-kulak: Building and Evaluating a 31.6M-Parameter {Turkish} Speech Recognizer from Scratch},
  author        = {Teke, Sait Furkan},
  year          = {2026},
  eprint        = {2611.03605},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2611.03605}
}

@misc{ufakzekakulak2026,
  title         = {ufakzeka-kulak},
  author        = {{ufak AI}},
  year          = {2026},
  howpublished  = {\url{https://huggingface.co/ufakai/ufakzeka-kulak}}
}

Links

• Model: https://huggingface.co/ufakai/ufakzeka-kulak
• Code: https://github.com/ufakai/ufakzeka-kulak
• Demo: https://kulak.ufakzeka.com
• Paper: https://arxiv.org/abs/2611.03605
• Write-up: https://ufakai.com/research/ufakzeka-kulak
• Contact: furkan@ufakai.com

Türkçe özet

ufakzeka-kulak, ufak AI’ın sıfırdan eğittiği küçük bir Türkçe konuşma tanıma modelidir. Apache-2.0 lisansıyla yayımlanmaktadır ve internet bağlantısı olmadan işlemcide çalışır. Aynı ağırlıklar NeMo, transformers, sherpa-onnx, parakeet.cpp (GGUF) ve parakeet-mlx (MLX) için bu repoda bulunmaktadır. Model büyük ve küçük harfleri ayırarak yazar ve noktalama işareti olarak nokta, virgül, soru işareti, kesme işareti ile kelime içindeki kısa çizgiyi kullanır; sayıları rakamla değil yazıyla verir. Yalnızca Türkçe konuşmayla eğitilmiştir ve tek seferde tek bir konuşma parçasını yazıya döker. Akış (streaming) modunda çalışmaz, yani konuşma sürerken metni kelime kelime yazmaz; konuşmacıları da birbirinden ayırmaz. Uzun kayıtların önce ses etkinliği algılamayla (voice activity detection, VAD) konuşma parçalarına bölünmesi ve her parçanın başında ve sonunda birkaç yüz milisaniyelik pay bırakılması gerekir; konuşmanın tam başladığı ve bittiği yerden kesilen bir parçada model büyük harfleri ve noktalama işaretlerini atlayabilir. Eğitim verisindeki her kaynağın lisansı ticari kullanıma izin vermektedir. YODAS kayıtlarında eğitim metinlerinin çoğunu bir konuşma tanıma modeli yazmış, ikinci bir model denetlemiştir; iki modelin üzerinde uyuştuğu hatalar eğitim metninde kalmaktadır. Yayın öncesi kabul testinde normalize edilmiş kelime hata oranı (WER) FLEURS Türkçe testinde 0,0921, Common Voice 27.0 Türkçe testinde 0,0916 olarak ölçülmüştür. Whisper base’in aynı kümelerdeki kelime hata oranları sırasıyla 0,2461 ve 0,3287’dir. İki model aynı ses kayıtları üzerinde karşılaştırıldığında farkın bootstrap güven aralığı iki kümede de tamamen sıfırın altında kalmış ve model kabul testini geçmiştir. Whisper large-v3’ün açgözlü kod çözmeyle kelime hata oranı ise 0,0604 ve 0,0767’dir ve ufakzeka-kulak’ınkinden düşüktür; bu fark için güven aralığı hesaplanmamıştır. Gündelik konuşmalardan oluşan Real-TurnTurk test kümesinde (6 konuşmacı) kelime hata oranı 0,3807 olarak ölçülmüştür. sherpa-onnx dosyalarında sesin örnek değerleri modele verilmeden önce 8 ile çarpılmalı, CTC dosyasında ayrıca sesin sonuna sıfır değerli örnekler eklenmelidir; ayrıntılar yukarıdaki sherpa-onnx bölümündedir. Apple M1 işlemcide sherpa-onnx ile tek iş parçacığında çalışan int8 TDT dosyasının on beş ile otuz saniyelik konuşma parçalarındaki medyan gerçek zaman faktörü (real-time factor) 0,0101 olarak ölçülmüştür, yani model bir saniyelik sesi saniyenin yaklaşık yüzde birinde yazıya döker; aynı bilgisayardaki Chrome’da, WebAssembly ile çalışan demo sayfasında bu değer 0,0311’dir. Doğruluk, konuşmacının cinsiyetine, yaşına ya da yöresel ağzına göre ayrı ayrı ölçülmemiştir. Konuşanların kimliğini belirlemek, onları izlemek, haklarında profil çıkarmak ya da bir kişiyle ilgili transkripti biri denetlemeden o kişi hakkında bir işlem için kullanmak bu modelin kullanım amacı dışındadır. Sonuçların tamamı, dosyalar ve bilinen sınırlar yukarıdaki İngilizce bölümlerde yer almaktadır.