EmbeddingGemma 2 Transcribe — LiteRT

This repository is a conversion of the audio encoder of EmbeddingGemma 2 and a small trained reader to LiteRT, made with litert-torch 0.10.0 for LiteRT 2.2.0 CompiledModel on Android: on a Galaxy S26 the encoder takes 156.7 ms for a 30 s window on the GPU at its default precision, against 344.6 ms at fp32, and the 54-minute recording transcribed from that output by the reader at fp32 comes out at a word error rate of 3.792 %, against 3.812 % (2026-10-31).

For English speech to text on an Android phone: the encoder, kept frozen, turns every 40 ms of audio into a vector, and the reader reads each vector as one of 200,000 English words or a blank. Audio, model and transcript stay on the phone.

The Android app that runs these files is eg2-transcribe on GitHub (android/). The reader's weights, training code and evaluation are in the companion repository mlboydaisuke/EmbeddingGemma-2-Transcribe-Reader on Hugging Face.

A browser demo is the Space mlboydaisuke/embeddinggemma-2-transcribe, built by Hugging Face's open-source team. It accepts up to 15 minutes of English speech, recorded in the browser or uploaded as a file. It does not run these LiteRT files: it runs the float32 PyTorch models they were exported from, with the same 30 s windows as the app, on a GPU.

The idea comes from Shahir's post (@TslShahir, 2026-10-13) and the Embedding Gemma Transcribe Mac app. This is an independent project, not affiliated with Google.

Files

On the phone. The app reads the two encoder files from its files directory and reader_w30.tflite, reader_w10.tflite and vocab.txt from the reader directory inside it; the app's README gives the adb commands, and the way to pick an encoder file on the phone instead.

Encoder. The audio tower and embed_audio of google/embeddinggemma-2 (revision 914f7f89), exported with litert-torch, one window length per file. Inputs: mel [1, F, 128], the log-mel spectrogram of transformers' Gemma4AudioFeatureExtractor (16 kHz audio, a 20 ms frame every 10 ms, 128 mel bins), and mask [1, F], 1 for a real frame; F = 3,000 for 30 s, 1,000 for 10 s. Outputs: soft [1, F / 4, 512], one vector per 40 ms, and mask [1, F / 4]. Every RMSNorm is rescaled so that its sums stay inside the float16 range, which the GPU at float16 and the NPU need; in float32 the rescaled graph gives the same numbers bit for bit. The weights of the fully connected and convolution layers are stored as float16 and turned back into float32 inside the graph, so a CPU, or a GPU at fp32 precision, computes in float32 with rounded weights.

Reader. Reader v2 (a 6-layer transformer of 19.6 M parameters trained with CTC on LibriSpeech's three training sets, 960 h; 890 h in training after leaving out the utterances with a word outside the vocabulary), the word table (the EmbeddingGemma 2 text embedding of each of the 200,000 words, float16 as released) and an argmax, in one graph. Inputs: soft [1, T, 512] and mask [1, T], with T = 750 or 250. Outputs per token: ids (int32, the best word), best (its score, 216.351 × the cosine to the word's text embedding) and blank (the blank's score). A token is its best word unless the blank scores higher, a run of the same word is one word, and blanks are dropped. The reader itself is float32; the table is used in float32 from its float16 values. Reader v2 keeps reader v1's inputs, outputs and file names, so the app runs either; reader v1 (4 layers, 13.3 M parameters, LibriSpeech train-clean-100) is this repository at revision 54d0e902.

Windows

The encoder runs on fixed windows and sees nothing before a window starts, so the tokens at the start of a window come out wrong. The windows follow the reader's contract: 30 s, 96 tokens (3.84 s) of overlap on the left, 1 extra token on the right.

• Window k starts at token 653 k (750 − 96 − 1). Window 0 keeps its tokens 0 to 748, every later window its tokens 96 to 748, and the window that reaches the end of the recording keeps through its last token.
• The log-mel is computed on the whole recording and each window is a slice of its frames. A window cut out of exactly 30 s of audio (480,000 samples) would give 2,999 frames, and its last token would miss its last frame.
• The reader reads the 750 tokens of a window as one block, both ways in time; the kept tokens of all windows make the transcript.

The encoder does 750 / 653 = 1.149× the work of one pass. The 10 s windows of the NPU keep 153 new tokens of 250. With these windows the app's word or blank equals the PyTorch reader's on all 81,436 tokens of the 54-minute recording below.

Performance

The Android app (debug build, LiteRT 2.2.0 CompiledModel) on a 54 min 17 s LibriVox recording (The Adventure of the Speckled Band, read by Kirsten Ferreri), in 125 windows of 30 s:

Pixel 8a: medians over the 125 windows; thermal status 2 from the second minute to the end of the run. The time covers log-mel, encoder, reader and words; reading the FLAC file came on top (5.4 s, the app's own FLAC reader). The app peaked at 5,267,760 kB of memory (VmHWM). The WER counts 377 errors in the 9,889 words of the book text, after lower-casing and collapsing spaces; the PyTorch reader on the same windows gives the same 3.812 %. The recording starts and ends with the LibriVox intro and outro, which the book text does not have, and a reader of an audiobook can depart from the printed words, so a WER on this recording is indicative only. The 8.1 s row is the app's bundled example (LibriSpeech test-clean 1284-1180-0000), which the app runs on the two 10 s files (a recording of up to 9.96 s): one of the 20 test-clean utterances of the accuracy check below, thermal status 1 at the end of that run. With reader v1 (this repository at revision 54d0e902) and an earlier build of the app, the same phone gave 1,401 ms and 839 ms per window, 276.8 s for the recording and a WER of 5.784 %.

Galaxy S26: medians over the 125 windows, in the same build of the app with the same five files (each file's SHA-256 on the phone equal to the table above). The run started on a rested phone (thermal status 0, battery 35.9 °C after a 9-minute wait) and the phone throttled as it ran: thermal status 1 at 25 s and 2 from 45 s to the end, the GPU's clock limit from 1,300 MHz down to 826 MHz and the fastest CPU cores from 4.7 GHz to 1.4 GHz by the end. Reading the FLAC file came on top (2.8 s). The app peaked at 4,472,048 kB (VmHWM). The WER is the same 377 errors, with the PyTorch reader's word or blank on all 81,436 tokens. On this GPU the reader's 30 s window costs 787 ms against 100 ms for its 10 s window (the 8.1 s row), so the app's choice of 30 s windows for a long recording is not the faster one on this phone. The 8.1 s row is the bundled example through the app's Try button, one run on a rested phone.

Galaxy S26, the NPU option (2026-10-17): the encoder on the Hexagon NPU in 532 windows of 10 s (96 tokens of overlap), the reader on the GPU on a thread of its own, reading window k while the NPU encodes window k + 1; the encoder waited 21.5 s in all for the reader, so the reader is the slower stage. Medians over the 532 windows; thermal status 0 at the start (after a 40 s rest) and 3 at the end; the app peaked at 3,665,264 kB. Reading the FLAC came on top (2.4 s), and a fresh install compiles the encoder for the NPU once before the first run (81 s). The WER counts 428 errors against the GPU fp32 run's 377. When the same 532 windows are run through the encoder and the reader in float32 on a computer, the count is 427 errors (4.318 %): the 10 s windows, which give the reader less context than 30 s windows, account for nearly all of the difference, not the NPU's float16 arithmetic. On the 20 test-clean utterances this form gives the reader's release transcripts for the 11 that fit one 10 s window; of the 9 longer ones, which it splits into 10 s windows, 4 differ. A screen recording of the same form on a rested phone, from the tap to the last word, took 73.7 s (44× real time).

The encoder alone on the Galaxy S26, measured with an earlier build of the app, before the reader was in (2026-10-13); one call writes the window, runs the encoder and reads the result:

The NPU computes in float16: over the full 54-minute recording, 37 of its 81,434 tokens fell under a cosine of 0.99 against the float32 encoder run on the whole recording at once (0.45 at the lowest), none of them digital silence. Token differences of that size do not by themselves raise the word error rate: the GPU at its default precision moves single tokens about as far from its fp32 output (a cosine of 0.45 to 0.46 at the lowest) and the transcript keeps its word error rate (below). The NPU is kept in the app as an option; GPU fp32 is the default. With the reader (the NPU row of the table above) this option reads the 54-minute recording in 71.1 s at a WER of 4.328 %, against 3.812 % for GPU fp32 in 140.8 s; its 10 s windows make the difference (above).

The encoder at the GPU's default precision (float16 arithmetic) against fp32 precision, on the 30 s file, in a test app on LiteRT 2.2.0 CompiledModel: one call writes a 30 s window, runs the encoder and reads the result; the median of 10 calls after 2 warm-up calls, each run in a new process. These are isolated calls in a test program, not the app's per-window medians in the table above, which were taken during the 54-minute runs.

On the 54-minute recording, the encoder at the GPU's default precision on either phone, read by reader_w30.tflite at float32 on a computer, gives a WER of 3.792 % (375 errors), against 3.812 % (377) for the same encoder file at fp32 on the same phone. The encoder's output does change: single tokens fall to a cosine of 0.46 (Galaxy S26) and 0.45 (Pixel 8a) against the fp32 output, and the reader picks another word or blank on about 0.1 % of the 81,436 tokens (97 on the Galaxy S26, 90 on the Pixel 8a), which leaves the word error rate where it was. FP16 with FP32 accumulation, the setting in between, gave 3.833 % (379) on the Pixel 8a. Keep the reader at fp32: at the default precision of a Mac's GPU, on the float32 encoder's output, it changed the transcripts of 2 of the 20 test-clean utterances and gave the 54-minute recording 383 errors. The app does not use this setting yet: GPU fp32 is its default, and its GPU default button runs the reader at the default precision too.

Accuracy

• Reader v2 alone (its release notes; float32 reference soft tokens, greedy decoding, no language model): WER 4.21 % on LibriSpeech test-clean, 10.42 % on test-other (reader v1: 6.81 % and 15.10 %).
• These files in the app on the Pixel 8a: on 20 LibriSpeech test-clean utterances, GPU fp32 and CPU both give 20 of 20 transcripts equal to the reader's outputs listed in its release notes, with the same word or blank as the PyTorch reader on all 5,036 tokens; on the 54-minute recording, the WER above, with the same word or blank as the PyTorch reader on all 81,436 tokens.

Known weaknesses

• Words that sound alike: 12.9 % of the reader's substitutions on test-clean are homophones (one pronunciation in the lexicon, such as "o" and "oh" or "meter" and "metre"), which a reader without a language model cannot tell apart. The most frequent substitutions are short function words: "and" and "in" for each other, "a" and "the" for each other, "an" read as "and".
• Long words: words of 6 or more tokens (240 ms or longer) are read wrong more often than short ones, 6.1 % against 1.8 % for words of 1 or 2 tokens (2,518 words of 100 test-clean utterances, word spans from a wav2vec 2.0 forced alignment).
• No punctuation and no capital letters; trained on read English audiobook speech (LibriSpeech) only.
• float16 arithmetic (the GPU's default precision, the NPU) moves some quiet tokens away from the float32 result. In the encoder that leaves the word error rate where it was (Performance); in the reader it changes words, so the reader stays at fp32. The app runs GPU fp32 unless another backend is picked.
• Memory: with the encoder and the reader both on the GPU the app peaked at 5,267,760 kB on the 54-minute recording and 5,142,000 kB on the 20 utterances (VmHWM, Pixel 8a, 8 GB); on the Galaxy S26 (12 GB) 4,472,048 kB and 4,343,156 kB. While the app loaded its models for the long recording, the phone's available memory (MemAvailable) fell to 873,848 kB; once, right after a long run on the CPU, it fell under 800,000 kB, where the device check stopped the app.

Conversion

The encoder files come from google/embeddinggemma-2 at revision 914f7f89142e33e77833254d9c9b90c3cef7303b; the reader files from reader v2 of mlboydaisuke/EmbeddingGemma-2-Transcribe-Reader at revision de224fd8fa4ed4519fdddd9aa4154432d46a1bd3 (reader.pt with SHA-256 d605f6fba8687497be6b5fd717ea6c51beaee5d5f69c30a64225da96ebe0d8c2), with the reader's model code from eg2-transcribe at commit 6bb4b7f. Both were converted with litert-torch 0.10.0 (main of google-ai-edge/litert-torch at 0a81bf6, installed editable: the released 0.9.4 lacks the Gemma 4 audio classes), transformers 5.19.0 and torch 2.12.0, and the float16 weights were stored with ai-edge-quantizer-nightly 0.10.0.dev20261013. The scripts and every step are in hf-to-litertlm scripts/eg2_transcribe/; one command per kind of file:

python scripts/eg2_transcribe/export_encoder.py export --model hf/embeddinggemma-2 --fixtures eg2-transcribe/reader/fixtures --out out/eg2
python scripts/eg2_transcribe/export_reader.py export --reader-dir hf/reader --reader-code eg2-transcribe/reader/train --fixtures eg2-transcribe/reader/fixtures --out out/eg2

On 2026-10-15 a rebuild of the two encoder files with these scripts, from the sources and versions above, gave them byte for byte, and the two reader files are the reader command's output from reader v2 the same day; the SHA-256 of all four are the ones in the Files table.

License and credits

• These files: Apache-2.0 (LICENSE, NOTICE); the app's code is Apache-2.0 as well.
• EmbeddingGemma 2 (Google DeepMind): Apache-2.0; use must follow the Gemma Prohibited Use Policy.
• LibriSpeech (the reader's training data and the test utterances): V. Panayotov, G. Chen, D. Povey, S. Khudanpur, "Librispeech: an ASR corpus based on public domain audio books", ICASSP 2015, https://www.openslr.org/12, CC BY 4.0.
• LibriSpeech lexicon (the words of vocab.txt): https://www.openslr.org/11, public domain.
• The 54-minute recording: LibriVox, The Adventures of Sherlock Holmes, section 8, read by Kirsten Ferreri (https://archive.org/details/adventures_holmes), public domain; its reference text is from Project Gutenberg eBook #1661 (public domain in the USA). Neither is in this repository.