Audio8-TTS-Realistic-0.6b — LiteRT

This repository is a conversion to LiteRT of Edge0/Audio8-TTS-Realistic-Air (renamed from Edge0/Audio8-TTS-Realistic-0.6b), made with litert-torch 0.9.4 and ai-edge-quantizer 0.9.0 for LiteRT 2.2.0: in the Android app under android/, a Pixel 11 Pro speaks an 18-token English sentence (5.48 s of audio) at a real-time factor of 0.853, and a Galaxy S26 at 0.797 (6.36 s of audio), each the median of 6 runs (2026-11-03). The real-time factor is the time from text to samples divided by the length of the speech: below 1, the speech is ready before it would have finished playing.

Edge0/Audio8-TTS-Realistic-Air is a text-to-speech model by Edge0, built on Audio8-TTS-Preview-0.6b. Its model card says that it supports 23 languages, and that it clones a voice zero-shot from a reference recording and its transcript or speaks without a reference. It uses a DualAR design inspired by Fish Audio S2 Pro: a 24-layer slow transformer predicts one semantic token per audio frame (2,048 samples, 46 ms at 44.1 kHz), a 4-layer fast transformer predicts the frame's 10 codec codebooks one at a time, and a neural codec turns the frames into audio. The transcript must match the reference recording exactly. A cloned voice speaking another language than its recording's: see Limitations.

This repository holds five LiteRT graphs (slow AR, fast AR, the codec decoder in two window sizes, the codec encoder), the source repository's tokenizer, two reference voices, a Python host loop for desktops and an Android sample app. The two autoregressive graphs hold int8 weights with float activations (dynamic-range quantization) and run on the CPU. In the app the codec decoder runs on the GPU; if the GPU's output fails a check at load, the same file runs on the CPU.

Quick start (Python, desktop)

pip install ai-edge-litert tokenizers soundfile scipy numpy huggingface_hub
hf download litert-community/Audio8-TTS-Realistic-0.6b --local-dir audio8r
cd audio8r
python audio8_tts_litert.py --model-dir . --text "Hello from LiteRT, fully on device." \
    --voice voices/en_librispeech_1272 --out hello.wav
# Japanese with the bundled Japanese voice
python audio8_tts_litert.py --model-dir . --text "今日は天気が良いので、公園まで散歩に行きましょう。" \
    --voice voices/ja_funasr_example --out konnichiwa.wav
# clone your own voice from a clip of up to 10 s and its exact transcript
python audio8_tts_litert.py --model-dir . --register-voice me.wav --ref-text "exact transcript of me.wav" --voice-out voices/me
python audio8_tts_litert.py --model-dir . --text "This sentence is spoken in my voice." --voice voices/me --out me_out.wav
# without a reference voice
python audio8_tts_litert.py --model-dir . --text "This utterance does not use a reference voice." --out noref.wav

The host loop runs every graph on the CPU through the ai-edge-litert Interpreter (--threads, default 4). It uses the sampler of the source repository's generation_config.json (temperature 0.7, top-p 0.9, top-k 50, at most 512 frames) and the vendor's repetition-aware re-draw; --temperature, --top-p, --top-k, --max-new-tokens and --seed change them. The source model card's own example samples at temperature 0.8, top-p 0.95, top-k 50. The host loop prefills a prompt in chunks through the smallest prefill signature that holds what is left of it (prefill_64 or prefill_256). XNNPACK packs the slow graph's weights separately for each of its three signatures, so the third one costs memory: on an Apple M4 Max with ai-edge-litert 2.2.0, generating the frames of four test sentences peaked at 2,074.7 MiB of resident memory, against 1,670.6 MiB with this repository's first slow file, which had no prefill_64 (2026-11-03). The Android app keeps one packed copy in a weight cache file (below).

Android

The app under android/ (Kotlin, LiteRT 2.2.0 CompiledModel API) speaks a typed sentence in one of the two bundled voices, or in your own voice: "Record my voice (10 s)" records you reading a sentence it shows, and that sentence becomes the voice's transcript. Both AR graphs run on the CPU with 4 threads; the slow AR updates its KV cache in place and prefills a prompt in chunks of 256 positions, a last chunk of up to 64 positions through prefill_64. At load the app creates and warms both GPU codec decoders, runs the 128-frame one on one test window and compares its output with the CPU output of the same file on the same codes (that CPU output is bundled in the app); the GPU decodes when the correlation is 0.99 or more, the CPU otherwise. A sentence may have up to 49 tokens. Build, install and file push steps: android/README.md. The app's code is under the MIT license (android/LICENSE).

Changes since this repository's first release (2026-11-02), measured in the app on both phones:

• Weight cache. Without a cache, XNNPACK packs the slow AR's weights again for each of the graph's three signatures at every load. The app writes the packed weights once to a file in the xnnpack/ folder of its cache directory (named by the SHA-256 of slow_ar_int8.tflite and the LiteRT version; 365,205,496 bytes on the Pixel 11 Pro, 365,259,896 on the Galaxy S26) and maps that file at every later load. Peak resident memory of the app process after load (VmHWM): 2.01 GB on the Pixel 11 Pro and 2.31 GB on the Galaxy S26, against 3.13 and 3.39 GB with the first release's slow file without a cache (2026-11-03). Each load hashes the 558 MB slow file first: 0.3 to 1.0 s on the Pixel 11 Pro, 0.2 s on the Galaxy S26. The first load after installing also writes the cache and compiles the GPU's programs: 23.9 and 45.4 s in two first loads on the Pixel 11 Pro. Android may delete the cache folder when storage runs low, and the next load writes the file again; if the slow graph cannot be created with the cache, the app creates it without.
• Prompt prefill. With a bundled voice, the prompts of the two test sentences below (231 and 236 positions) are one 256-position chunk. A voice recorded in the app makes the prompt longer, and the part past 256 positions now runs as one prefill_64 call instead of a padded prefill_256 call. For an English sentence of 12 tokens (283 positions) and a Japanese one of 48 tokens (319 positions) in a recorded voice, the prompt prefill took 235 and 233 ms on the Galaxy S26 against 339 and 330 ms, and 514 and 745 ms on the Pixel 11 Pro against 972 and 1,163 ms (medians of 6 runs, 2026-11-03). In each Pixel 11 Pro process (12 sentences, in both forms), the slow AR slowed from about 12 to 16–27 ms per frame from the 4th, 5th or 6th sentence on, under CPU frequency caps.
• Load. The app creates and warms the 192-frame GPU decoder at load, and checks the GPU's output against the bundled CPU output instead of decoding the test window again on the phone's CPU. With the GPU's compiled programs on the phone, load took 8.71 s on the Pixel 11 Pro and 6.92 s on the Galaxy S26, against 19.21 and 10.86 s with the first release's load path (means of 2 app processes, 2026-11-02). On the Galaxy S26 the English test sentence (137 frames) no longer creates the 192-frame decoder in the first run of a process: its codec decode took 1.40 and 1.41 s there, against 2.38 and 2.30 s.
• Frame cap. When a sentence stops at the frame cap, the app's message gives the cap in frames and seconds as set (512 frames, 23.8 s, by default) and its log gets a FRAME_CAP line; the host loop prints a [cap] line.

Files

The codec checkpoint and the codec code are the same files in the Realistic and Preview source repositories, so the codec graphs come from litert-community/Audio8-TTS-Preview-0.6b: the encoder unchanged, the two decoders rewritten. The two voices are that repository's files.

Codec decoder on the GPU

The two codec decoders as first published in the Preview repository gave a wrong output on the LiteRT GPU accelerator: on a Pixel 11 Pro and a Galaxy S26 (OpenCL, LiteRT 2.2.0 and 2.3.2) and on an Apple M4 Max (Metal), at the GPU's default precision (float16) and at float32. The Preview repository has carried the rewritten decoders since 2026-11-02 (commit 3d59a74c). The decoders here carry three graph changes:

• Before two MUL operators whose non-constant operand (computed at run time) has rank 2 and the output rank 3, a RESHAPE gives the operand the output's shape. The GPU accelerator lays such a rank-2 tensor out differently from a rank-3 one, and the MUL then multiplied the wrong elements, at float32 too. The RESHAPE keeps the values and their order.
• Each RMSNorm squares its input scaled by 1/16, with epsilon scaled by 1/256 and the result scaled back. Unscaled, one square reaches 121,972.6 and the 1,024-channel sums of squares about 76,500, both more than float16 holds (65,504), and the output stops depending on the codes. Powers of two only move the exponent, so the CPU result is the same.
• The attention mask's −1e9 becomes −30000. In float16 −1e9 is −inf, and in the 192-frame decoder a row with no finite score turned into NaN.

On an Apple M4 Max CPU, the rewritten decoders give the original's output byte for byte on all 32 test code sequences (15 for 128 frames, 17 for 192). On the GPUs of both phones at the default precision, their output correlates with that CPU output at 0.9998 or more on every speech test sequence, and the Pixel 11 Pro run in Accuracy decodes on the GPU. The rewrite is codec_gpu_rewrite.py in hf-to-litertlm (audio8_tts_work/).

Accuracy

The reference is the vendor's own code at revision 1e857568 in float32 on the CPU (PyTorch 2.12.1, transformers 4.57.6), with the sampler of generation_config.json and seeded draws. The test set has 30 sentences in six of the 23 languages: 6 English and 6 Japanese with a reference voice of the same language; 3 each in Arabic, Portuguese, Russian and Turkish without a reference, and 1 each of those four with the English reference voice; one English and one Japanese without a reference. Set B repeats set A with every sampling seed raised by 100,000. Each output was transcribed by whisper large-v3-turbo and compared with its text (errors counted in words, in characters for Japanese), compared with the reference clip by TitaNet-L (speaker cosine), and scored by UTMOS22.

The errors are out of 94 English words, 162 Japanese characters, 33 Arabic, 41 Portuguese, 31 Russian and 28 Turkish words per set; N below is that count. Two pass rules:

• First rule, written before any of these numbers, on set A alone: per language, at most the reference's errors plus max(2, ⌈0.03 × N⌉), and the speaker, UTMOS22 and end-of-speech checks of the next rule, taken against reference set A. The Mac CPU run failed it on English (6 errors against a bar of 4) and passed its other checks; the phone's set A errors are within its bars.
• Rule applied here, written after that result and before any set B run of these graphs: per language, the errors of sets A and B together at most the reference's A and B errors plus max(2, ⌈0.03 × 2N⌉) (bars: en 10, ja 10, ar 12, pt 9, ru 3, tr 4); on set B, each speaker group's mean cosine at least the reference's minus 0.05, the UTMOS22 mean at least the reference's minus 0.10, at most 21 of the 30 sentences more than 0.02 below the reference's UTMOS22 on the same sentence, and every sentence ended by end of speech. The Mac CPU run passes it (Arabic errors 12 against a bar of 12), and so does the phone (Arabic errors 8 against a bar of 12).

The Mac run used ai-edge-litert 2.2.0 (Interpreter, 4 threads) on an Apple M4 Max, the slow graph at a KV cache of 2,048 positions and the Preview's codec decoders. With a KV cache of 1,024 positions, the slow graph gives bit-identical logits and hidden states on the 38 prompts tested (the 30 above and 8 more), and the rewritten decoders give byte-identical CPU output. The phone run is the app with the files of this repository's first release, the codec on the GPU and the slow AR's KV cache in the app's ping-pong mode (two buffer sets, each call reading one and writing the other); updating that cache in place gives the same codes on both phones (the two speed-test sentences, every run).

The slow file of this release is the same export with a prefill_64 signature added. On the Mac CPU its prefill_256 and decode give bit-identical outputs to the first file's on the 38 prompts; a prompt prefilled as 256 positions plus one prefill_64 call leaves the same KV cache and logits as 256 positions plus a padded prefill_256 call (12 of 12 runs), a prompt of up to 64 positions gives the same outputs through prefill_64 as through a padded prefill_256 (4 of 4), and the host loop makes the same codes with either file (4 of 4 sentences). In the app, the current file read from its weight cache made the same codes as the first file without a cache (12 of 12 runs on each phone).

Not shipped, measured on the same test: int4 weights in the slow AR. Four int4 recipes made an AR frame (one slow decode, ten fast steps) 4.6 to 15.2 % faster in our runner on the two phones on one thread, and none passes the rule above on the Mac CPU. The first (int4 in blocks of 32 on every linear layer) fails it on Turkish (8 errors against a bar of 4), Arabic (14 against 12) and UTMOS22 (3.834 against 3.864). The other three, each tested as a file with this repository's int8 prefill and its own int4 decode, fail it on set B's UTMOS22 (3.750, 3.801 and 3.850 against 3.864). The Preview repository's int8 codec decoder lowers UTMOS22 from 3.904 to 3.690 on the same codes, on every one of the 30 sentences.

Performance

Measured with LiteRT 2.2.0 on a Galaxy S26 (SM-S942Q, SM8850, Android 16) and a Pixel 11 Pro (Tensor G6, Android 17).

In the Android app

The files of this repository: both AR graphs on the CPU (4 threads), the slow AR's KV cache updated in place and its weights read from the weight cache, the codec decoder on the GPU. Each sentence was spoken 6 times, 3 in each of two app processes (2026-11-03).

• Sentences: "Our train departs early in the morning, so we should reach the station on time." (18 tokens, the English voice, seed 1004) and "この製品は、スマートフォンの上で音声を合成することができます。" (17 tokens, the Japanese voice, seed 1001). Each value except the range is the median of the 6 runs.
• Text to samples covers building the prompt, the prefill, the frame loop and the codec decode; playback starts after it.
• With the same files and seed, the English sentence came out as 137 frames on the Galaxy S26 and 118 on the Pixel 11 Pro: the two phones' CPUs give slightly different logits for the int8 graphs, and sampling follows them. Each phone gave the same codes in all 6 runs.
• The Galaxy S26's real-time factors are higher (slower) than in this card's first release (0.725 and 0.655): in 10 of the 12 runs the slow AR took 10.1 to 11.0 ms per frame instead of 8.7 (CPU frequency caps), and the first release's slow file without the weight cache, run in the same sessions, gave 0.798 and 0.753. On the Pixel 11 Pro the slow AR took 11.7 to 12.5 ms per frame in all 12 runs, and the first release's slow file gave 0.854 and 0.875 in the same session.
• Load (the AR graphs, the slow AR's weights read from the weight cache, both GPU decoders built from the GPU's compiled programs kept on the phone, and the output check): 6.71 and 6.42 s on the Galaxy S26, 8.21 and 8.27 s on the Pixel 11 Pro. Peak resident memory of the app process (VmHWM): 2.31 GB on the Galaxy S26 (2,256,996 to 2,257,604 kB), 2.01 GB on the Pixel 11 Pro (1,961,788 to 1,962,316 kB); GB = kB × 1,024 / 10⁹.
• Every process started at thermal status 0 without a CPU frequency cap. The Galaxy S26 ended both processes at status 2 with caps; the Pixel 11 Pro ended both at status 0, the second with a cap on one CPU cluster.

One call of each graph

Our benchmark runner on the same phones, LiteRT 2.2.0 CompiledModel API. A value is the median time per call (writing the inputs, running, reading the first output back; the AR graphs' KV state stays on the device), with the number of timed calls in parentheses. CPU: XNNPACK, 4 threads, read from the runner's process in every run. GPU: the LiteRT GPU accelerator (OpenCL) at its default precision (float16). NPU: the Galaxy S26's NPU, through graphs compiled ahead of time for SM8850 with QAIRT 2.47. Our runner's CPU rows and the GPU rows of the two codec decoders were measured on 2026-11-08, the other rows on 2026-11-01 and 2026-11-02.

In this card's earlier versions the CPU rows of this table ran on one thread, not four: our runner passed the thread count under an option name that LiteRT 2.2.0 does not read.

On the Galaxy S26 the GPU rows of the two codec decoders are the first calls from a rested phone: one call on one input, then three timed calls, in each of two runs. In longer runs that phone lowered its CPU clock limits before the timed calls ended, and in the first of the two T192 runs it lowered a clock limit 4.1 s after the graph was ready, so that row counts the four timed calls that ended before it. On the Pixel 11 Pro they are eight timed calls on one input in one run, after one call on each of the 15 (T128) or 17 (T192) test inputs and two warm-up calls, without a CPU clock limit; in the same session the first calls from the rested phone took 1,153.2 and 1,660.4 ms (6 and 2 calls). The encoder row has no value on either phone: on four threads each phone lowered the clock limit of its large cores before the first timed call ended, in all four runs (0.1 to 1.4 s after the graph was ready on the Galaxy S26, 5.9 to 11.7 s on the Pixel 11 Pro). The CPU row of codec_decoder_fp16_T128.tflite is the app's own call on real codes, one per app process after a warm-up call (2026-11-02). This card's first release listed our runner's CPU calls of the Preview's float16 and int8 T128 decoders instead; the phones' CPU frequency was capped while they ran, so they are left out.

A frame takes one slow decode call and ten fast step calls. At a KV cache of 2,048 positions that is 12.38 + 10 × 1.099 ms on the Galaxy S26's CPU against 21.0 + 10 × 3.683 ms for the GPU-rewritten AR graphs (not shipped), and 19.96 + 10 × 7.252 ms on the Pixel 11 Pro's CPU against 53.2 + 10 × 19.935 ms. On one thread our runner's step took 1.01 ms on the Galaxy S26 and 2.04 ms on the Pixel 11 Pro, against 1.10 ms and 7.2 to 8.0 ms on four threads in the same sessions (medians of 50 calls in each of two runs, 2026-11-08); in the app, on four threads, the fast AR took 13.16 to 15.77 ms per frame of ten calls (the app table above). We did not look into why the runner's four-thread step is slower than the app's on the Pixel 11 Pro. In a separate session on the Galaxy S26, the per-call medians of a frame added up to 44.7 ms for those graphs compiled for the NPU and 24.4 ms for the CPU graphs on one thread. So the AR graphs here run on the CPU, at a KV cache of 1,024 positions. A GPU prefill with a CPU decode was measured for this release and not shipped: the GPU-rewritten slow AR's prefill_256 at a KV cache of 1,024 positions, with its 48 KV tensors read back for the CPU graph, added up to 85.7 ms on the Galaxy S26 (medians: GPU body 75.9 + KV read back 6.8 + embedding 3.0 ms) and 327.9 ms on the Pixel 11 Pro (177.0 + 144.6 + 6.3 ms), against 220.5 and 607.4 ms for the CPU prefill_256 on one thread in the same sessions (142.7 and 336.9 ms on four threads, in the table). The app's CPU prompt prefill was 175 and 327 ms in this card's first release, so the GPU path would save at most 89 ms per prompt on the Galaxy S26 and nothing on the Pixel 11 Pro.

NPU (our runner, 2026-11-02 and 2026-11-03). We found no NPU form to ship. On the Galaxy S26, with graphs compiled ahead of time for its NPU:

• AR graphs: the slow AR's prefill_256 (160.2 ms in the table) was faster than on the CPU but slower than on the GPU, whose prefill this release does not use either (above), and a frame was slower than on the CPU (above). One graph holding a frame's ten fast steps and the sampler took 19.76 and 19.26 ms per frame (medians of 30 calls in each of two runs) against 9.48 and 9.55 ms for ten steps of the shipped CPU graph on one thread without the sampler, which the app runs in Kotlin (10 × 1.099 ms on four threads, in the table); its codes differed from the CPU's in 29 of 76 test frames (the NPU compiler ran its float operations at float16; the same graph on the phone's CPU gave the CPU's codes in all 76). The AR graphs were not run on the speech test above.
• Codec decoder: the float16 T128 decoder took 1,742.6 ms against 946.2 ms on the GPU in the same session and lowered UTMOS22 on all 14 test sentences. Of four int8-weight versions of it (none shipped), the compiler refused the first (per-channel int8 weights with float compute) and the five int16 × int8 transposed convolutions of the second (they would have run on the CPU); our runner could not read the third one's int16 output; the fourth, compiled whole for the NPU, gave a wrong waveform (correlation −0.04 to 0.04 with the same graph on the Mac CPU) in 1.91 to 1.94 times the GPU's time.

On the Pixel 11 Pro (LiteRT 2.3.0 and its NPU plugin for the Tensor chip, which compiles on the phone), the compiler refused the T128 decoder with float16 weights and with int8 weights (UNIMPLEMENTED: DeduceInputSlice not non-identity InternalPadding not yet implemented.); we did not try ahead-of-time compilation for it.

Limitations

• The prompt (reference transcript, reference codes and text) plus the generated frames must fit the slow graph's KV cache of 1,024 positions. The host loop refuses a longer prompt and stops when the cache is full; the app refuses a sentence over 49 tokens, and its longest prompt (49 tokens and a 10 s voice recorded in the app) is 320 positions with the English recording script and 324 with the Japanese one, which leaves room for the 512-frame cap.
• At most 512 frames (23.8 s of speech) per call by default; the host loop stops there and prints a [cap] line, and the app says so in its message (an 80-token English text reached the cap in 2 of 2 runs on the Mac and lost its last sentence).
• The codec decoder is causal, but its transformer works on 128-frame windows, so decoding in windows is not sample-exact against one offline decode of the whole utterance. The host loop and the app make one 128-frame call up to 5.9 s of speech, one 192-frame call up to 8.9 s, and beyond that 192-frame windows with 128 frames of left context.
• Voice registration takes up to 10 s of reference audio (the encoder has one fixed length), and the transcript must match the audio.
• Record the reference voice in the language it will speak. With the bundled English voice, 9 of the 10 other languages tested had a sentence with two or more word errors (Arabic, Finnish, Hindi, Indonesian, Norwegian, Swedish, Thai in characters, Turkish, Vietnamese; Malay had none), 21 of the 30 sentences in all, and the source model's own code had 17 of 30 on the same sentences and voice. With a Swedish reference voice, 15 of 15 Swedish runs (5 sentences, 3 seeds) had no word errors. Measured on the Mac with the host loop and this repository's first files (2026-11-02).
• The accuracy test covers 6 of the 23 languages (English, Japanese, Arabic, Portuguese, Russian, Turkish). The other 17 were not tested on these files.
• Speed was measured on one Galaxy S26 and one Pixel 11 Pro with our runner and app; other devices were not measured. The Mac numbers are agreement checks, not speed.
• From the source model card: quality varies by language; noisy reference audio or a mismatched transcript can reduce stability and speaker similarity; speech may contain pronunciation errors, repetitions, omissions or early stopping, so listen to it before using it. Obtain permission to clone a voice, and disclose synthetic audio where appropriate.

Reproduce

Source: Edge0/Audio8-TTS-Realistic-Air (the old name Edge0/Audio8-TTS-Realistic-0.6b redirects to it) at revision 1e857568bfd4f02931aca3988ee773a44d2d26d3; the three later commits on its main branch (925bbd4c on 2026-11-01, 7c82ad18 and 23c305cc on 2026-11-02) change only README.md and assets/realistic-hero.svg. The export, quantization and decoder rewrite commands are in the Audio8-TTS-Realistic-0.6b section of REPRODUCE.md in hf-to-litertlm, with the scripts under audio8_tts_work/. Run again on 2026-11-03, they rebuilt slow_ar_int8.tflite, fast_ar_int8.tflite and both codec decoders with the SHA-256 of the files here.

License

Apache-2.0, inherited from the source model by Edge0 (LICENSE, a copy of the source repository's file). Attribution is in NOTICE. The Android app under android/ is under the MIT license (android/LICENSE).

Changes from the original work. The slow and fast AR transformers were exported from the checkpoint's weights to LiteRT graphs with the KV cache as graph inputs and outputs, and their weights quantized to dynamic int8. The codec graphs come from the Preview conversion; the decoders' graphs were rewritten as described above. The tokenizer is unchanged. The host loop and the app port the vendor's prompt format and sampler.