QUAN-Transcribe-1.0 (Core ML)

QUAN-Transcribe-1.0, a fast Vietnamese speech recognition model, converted to Core ML to run on Apple devices. The Python package in this repo does not need PyTorch.

• Same accuracy as the PyTorch model: on FLEURS vi_vn, word error rates match to within 0.02 points (table below).
• About 270× faster than real time on an M3 Max GPU (fp16).
• Every feature of the original: files and waveforms, long recordings, streaming mode, live microphone transcription, optional KenLM language model, CLI and Python API.

Quick start

Requirements: macOS 15 or later, Python 3.10 or later. Apple Silicon is recommended.

hf download qnguyen3/QUAN-Transcribe-1.0-CoreML --local-dir QUAN-Transcribe-CoreML --exclude "QuanTranscribe-fp32.mlpackage/*"
cd QUAN-Transcribe-CoreML
uv sync                                    # or: pip install .
uv run quan-transcribe-coreml --model . audio.wav
uv run quan-transcribe-coreml --model . --mic

Without --model ., the files are downloaded from this repo into the Hugging Face cache, and only the precision you use is fetched.

from quan_transcribe_coreml import QuanTranscribeCoreML

asr = QuanTranscribeCoreML.from_pretrained("qnguyen3/QUAN-Transcribe-1.0-CoreML")
print(asr.transcribe("audio.wav"))
print(asr.transcribe("audio.wav", streaming=True))

# waveforms and lists work too
texts = asr.transcribe([wav_a, wav_b], sample_rate=16000)

• Input: any file libsndfile can read (wav, flac, ogg, mp3, ...), or a numpy waveform. Audio is converted to 16 kHz mono.
• Output: lowercase Vietnamese without punctuation, numbers written as words.
• Long audio: recordings are cut into pieces of at most 30 s at quiet points (chunk_seconds).
• First run: the model is compiled once (a few seconds) and cached in ~/.cache/quan_transcribe_coreml. Each input length then loads in about a second the first time it is used. asr.warmup() loads all of them up front.

CLI

quan-transcribe-coreml audio.wav
quan-transcribe-coreml --streaming audio.wav
quan-transcribe-coreml -v *.mp3                 # several files, with duration and speed
quan-transcribe-coreml --mic                    # live from the microphone, Ctrl+C to stop
quan-transcribe-coreml --lm audio.wav           # beam search with the n-gram LM

Streaming mode

streaming=True decodes the way the model runs in streaming mode. The reader's attention covers only the current 0.48 s chunk and earlier ones. The encoder's attention is already chunked with left context only, and the reader's convolution is causal, so nothing looks more than one chunk ahead.

Live microphone

--mic shows the sentence in progress, dimmed and updated every 0.48 s. Streaming mode keeps the text from finished chunks fixed, so only the newest words can change. After a pause (--endpoint, default 0.8 s) the sentence is decoded once more with full context (and with the LM when --lm is given) and printed on its own line. All model lengths are loaded before listening starts, so no update stalls later. When stdout is not a terminal, only finished sentences are written.

from quan_transcribe_coreml.live import LiveSession

session = LiveSession(asr)
for block in blocks:                          # 16 kHz mono float32 audio, any block size
    for sentence in session.accept(block):    # finished sentences
        print(sentence)
    print(session.partial)                    # sentence in progress
print(session.flush())

n-gram language model

A 4-gram KenLM model (lm/, 91 MB) can be used for CTC beam search. It lowers WER by about 2 points.

uv sync --extra lm                            # or: pip install ".[lm]"  (builds kenlm; needs a C++ compiler)

asr = QuanTranscribeCoreML.from_pretrained("qnguyen3/QUAN-Transcribe-1.0-CoreML", lm=True)
asr.transcribe("audio.wav")                   # beam search with the LM
asr.transcribe("audio.wav", use_lm=False)     # greedy

The LM was trained on about 29M words of Vietnamese news and Wikipedia sentences from the Leipzig Corpora Collection (news 2022, Wikipedia 2021), plus training transcripts. It is the same LM as in the PyTorch repo.

Accuracy and speed

FLEURS vi_vn validation, the 317 utterances whose reference has no digits (61 minutes of audio), same text normalization and scoring for every row:

Speed on an Apple M3 Max with greedy decoding, models warmed up, from waveform to text:

Use fp16 on the GPU. On the CPU, fp16 arithmetic costs accuracy, so compute_units="cpu_only" loads fp32 by default. The Neural Engine cannot compile this model, so all and cpu_and_ne fall back to the GPU and CPU after a long first load. Use the default cpu_and_gpu.

Files

Using the model from Swift or other Core ML code

Each .mlpackage is a multifunction model (macOS 15 / iOS 18 or later). There is one function per input length, len_2s, len_4s, len_6s, len_8s, len_10s, len_15s, len_20s, len_25s and len_30s, all sharing the same weights. Pick the shortest one that fits the audio and zero-pad up to its length.

Log-mel features and all attention masks are computed inside the model. To get text, take the argmax per frame, merge repeats, drop unit 0 (blank) and 1 (unknown), join the pieces from units.json and replace ▁ with a space. Set the model's compute units to CPU and GPU.

How it was converted

convert/convert_coreml.py traces the PyTorch model one input length at a time with coremltools 9.0 and merges the lengths into one multifunction package. Some changes keep the converted model exact and safe in fp16:

• The log-mel spectrogram is a strided convolution with DFT weights, so the model takes raw audio.
• Masks are built from the length input with integer math (sample positions overflow fp16 beyond about 4 s) and use a finite -10000 instead of -infinity, so padding rows never turn into NaN.
• Inputs to layer norms and RMS norms are divided by 64 first. Activations reach about 1,200, and their squares would overflow fp16. Normalization is unchanged by scaling, apart from its tiny epsilon.

To reproduce: pip install ".[convert]", then python convert/convert_coreml.py --out ..

License

This model is a derivative of EmbeddingGemma 2. It contains the EmbeddingGemma 2 audio encoder weights and is subject to the Gemma Terms of Use and the Gemma Prohibited Use Policy. See NOTICE.