MOSS-TTS-Nano 100M for Core ML on the Neural Engine

A Core ML re-export of MOSS-TTS-Nano-100M and the decoder of MOSS-Audio-Tokenizer-Nano (both Apache-2.0, by the OpenMOSS team) that runs with MLComputeUnits.cpuAndNeuralEngine, without the GPU, so it keeps working in an iOS app playing in the background.

Sources: model 44502f80dbf9743528fa921cc544d662c685ebec, codec 6aa02b01e445cc585582cf0ba480bc3ea6c8dd68.

Differences from the upstream graphs

• No MLState. Every graph is static and stateless; the host owns the KV caches and passes them as inputs. Prompt lengths and contexts are buckets.
• All fp16. RoPE angles come from the host in float64, so phases do not accumulate fp16 error.
• The local decoder's 17 steps per frame are one call, with top-k, top-p, temperature and repetition-penalty sampling in the graph (Gumbel-max with noise from the host).
• The codec decodes 12 frames per call with its sliding-window caches as inputs and outputs.

Files

Each model is present as .mlpackage and as .mlmodelc (compiled for iOS 26). Load each function with MLModelConfiguration.functionName. config.json holds the buckets, token ids, sampling defaults and quality levels; tokenizer.model / tokenizer.json are the upstream SentencePiece tokenizer.

Measurements (Apple M5 Max, macOS 27, .cpuAndNeuralEngine)

Real-time factor with the full loop in Swift: 0.069 (Portuguese) and 0.066 (English) with 16 codebooks.

Whisper large-v3-turbo word error rate on a test sentence:

• English: 0.00 for both PyTorch and Core ML.
• Portuguese: 0.25 for PyTorch and 0.23 for Core ML, averaged over 6 seeds.

The Portuguese output tends toward a European accent even with Brazilian voice prompts. The upstream model does the same.

Voices

A voice is a prompt of codec codes, in voices/<name>.json, next to the clip it was encoded from.

Generation loop

1. Build the prompt rows (17 int32 each: a text id and 16 codes) from config.json's prompt strings, the voice codes and the sentence's SentencePiece ids.
2. Run prefill_P for the smallest bucket that fits, then for each frame:
   • call frame_D to sample the frame's codes and end flag;
   • call step_C with the new row to get the next hidden state and the new key and value;
   • every 12 frames, call chunk_12 to decode audio.
3. Output is 48 kHz stereo, 12.5 frames per second.

The quality levels in config.json combine codebook depth (12 or 16) with a retry budget that redraws takes whose length falls outside 1.8–4.2 frames per text token.

License

Apache-2.0, as the upstream model and codec. The LibriVox clips are in the public domain.