kale-tts-1-nano is a very small English text-to-speech model with 10 voices designed with Maya-1, that runs on CPU or GPU on low-end devices. It stays as close as possible to Maya-1's melody, which already sounds great, while running on weak hardware, in one 14.8 MB file (the CPU file).
| measure | value |
|---|---|
| size | 14.8 MB on the CPU (int8, kale-tts-1-nano.onnx); 30.6 MB for WebGPU in the browser (float32, kale-tts-1-nano.fp32.onnx) |
| parameters | 7.5M |
| unclear words | +1.6 points vs Maya-1's own recordings (Maya-1: 5.82 %) |
| melody gap | +7.46 points vs Maya-1's own recordings (Maya-1: 0) |
| speed, Samsung Galaxy Tab S10 Lite CPU | 44x real time, first sentence after 0.09 s |
| speed, Mac browser | 151x (WebGPU), 82x (WASM) |
Unclear words: a speech recognizer listens word by word, and "unclear" is the share of words it isn't sure about; Maya-1's own recordings score 5.82 % on the same lines. Melody gap: how far the voice's pitch melody is from Maya-1's own recordings of the same lines, in percentile points; Maya-1 scores 0. Lower is better for both.
Want better quality in a bit bigger file? Try kale-tts-1-tiny.
Trained on kale-tts-1-data: 204.7 hours (99,783 takes) of the 10 voices' speech with its text, CC BY 4.0, open for anyone to train their own.
| name | voice | pronunciation |
|---|---|---|
| Kenzie | esports hype woman | US English |
| Simone | deep and confident woman | US English |
| Amy | crisp news-style woman | British English |
| Frankie | hard-boiled detective's partner | US English |
| Darlene | rough low-voiced old trucker woman | US English |
| Owen | bright young host | US English |
| Toby | mischievous prankster | US English |
| Jack | crisp news-style man | British English |
| Earl | African American jazz granddad | US English |
| Magnus | deep dramatic trailer narrator | US English |
Every voice was designed from a one-line text description; none is a real person.
Every voice, in your browser: the kale-tts-1 demo.
We build a privacy-focused computer for kids. Our goal is to run all the AI locally on the device, so privacy is as high as it can be. Audio models were the quick first win, because they're easy to do with AI right now.
We made our own voice for Kale (Hazel), and we're keeping that one. But we share the whole concept, so you can recreate any voice from Maya-1. And here are 10 voices to try right away.
7.5M parameters in one 14.8 MB file (the CPU file; 30.6 MB for WebGPU): the dictionary in front, a small part that reads the text and plans timing and melody, a sound generator, and a voice pack per voice.
No flow matching and no diffusion anywhere: one pass per sentence.
Tech terms first ("Node.js" -> "Node J S"), then numbers, dates, money and units as words (dizione rules), then every word through a
CMUdict-based dictionary, a hand-checked override list and a small guesser for unknown words. No espeak. Files: frontend/.
Small TTS models live or die by their front end: the wrong sounds make the best voice say the wrong word. We checked every word the front
end can meet (334,000 words, American and British) and fixed 3,800+ readings in frontend/words/override.public.tsv. Example: "details"
came out of the base dictionary as dih-TAILS (dɪtˈeɪlz); one line in the override list makes it DEE-tails (dˈiːtˌeɪlz).
Add a word: one line in frontend/words/override.public.tsv (British: frontend/words/en-gb/override.public.tsv): the word in
lower case, a tab, its IPA (stress mark right before the vowel). It wins over the dictionary and the guesser. Prompt for a coding agent:
In this repository, add the word "<word>" to frontend/words/override.public.tsv with the pronunciation <IPA> (one line: word, a tab,
the IPA in the same style as the other lines). Check that the word is not a person's name and is not offensive; if it is, stop. Run the
front end on a sentence with the word (python/kale_tts.py: QuickJSFrontEnd) and confirm the new IPA. Then render it with kale-tts-1-nano and listen.
A compact student turns the phonemes into how long each sound lasts, its pitch and its loudness. It learned this from kale-tts-1-base, on text only.
A small generator (3.7M): a harmonic source plus noise through a learned filter, then an inverse STFT to 24 kHz. No separate vocoder, no flow matching: one pass per sentence. At the end of a sentence a per-voice pitch floor keeps the last words from creaking.
Each voice starts as a one-line description given to Maya-1. Maya-1 reads thousands of sentences in that voice; a filter keeps the takes
that read back right, match the voice and end cleanly. Each voice ships as a style table and a 16-number sound row (voices/).
99,783 original Maya-1 takes, 204.7 hours, 10 voices, texts from public-domain books: kale-tts-1-data (CC BY 4.0). About 25 hours of clean takes per voice is where clarity levels off; 5 hours already sounds right but is less clear.
We tried open-weight models on the same 192 test lines against Maya-1's own recordings. Fine-tuned on Maya-1's audio, Kokoro-82M was the clearest and the closest to Maya-1's melody, but it ran at about real time on the tablet, far too slow, so it became the teacher: kale-tts-1-base. The published kale-tts-1-base keeps that fine-tune's text side and planner with stock Kokoro-82M's sound decoder; on the same bench it is less clear (+3.9 points) than tiny and nano.
192 English lines (stories, explanations, numbers, questions), all 10 voices, sentence by sentence as an app speaks; each voice against
Maya-1's own recordings of the same lines. Unclear words: word spans from CTC forced alignment (wav2vec2-large-960h-lv60-self); kale-tts-1-nano
reads 7.47 % unclear, Maya-1's recordings 5.82 %. Per voice: results/bench/bench-10voices.json.
Export check: every release file against a float32 build of the same weights on 200 sentences; the same length on each (int8: at most one
sentence one frame off) and a log-mel distance of at most 0.20. nano: the same length on all 200 sentences, log-mel distance 0.091; the float32 file is identical to its reference. Whisper reads all 10 voices back right (2.22 % word errors on the check lines, 0.53 % on the 192 bench lines).
recipe/render/, recipe/filter/), then teacher, heads and a new sound row (recipe/train/, recipe/export/).| where | runs | speed and notes |
|---|---|---|
| Browser, WebGPU | yes | 151x; loads the 30.6 MB float32 file (onnxruntime-web 1.31 or newer) |
| Browser, WASM | yes | 82x with 4 threads (the 14.8 MB CPU file; threads need COOP/COEP headers) |
| Android (onnxruntime) | yes | Galaxy Tab S10 Lite CPU: 44x, first sentence after 0.09 s |
| iOS | not tested | onnxruntime runs there; we have no device numbers |
| Python | yes | 240x (python/kale_tts.py) |
| Node.js | yes | 251x (js/kale-tts.mjs) |
HTTP server (server/server.mjs) |
yes | 152x for whole lines, text in, WAV out (one process; WAV or streamed) |
| transformers.js | yes | with our text-prep helper (see Use it) |
| sherpa-onnx, kokoro-onnx | coming | the kokoro-compatible export is being rebuilt for this version |
| C, Go, Rust, Swift | yes | onnxruntime C API, onnxruntime_go, the ort crate, the Swift package |
Speed = times faster than real time, median over 70 test lines in Kenzie's voice (18 Nov): the model one sentence at a time; the server row whole lines, text in, WAV out. Machines: Apple MacBook Pro, M5 Max (18 CPU cores, 128 GB RAM) for the browser, Python, Node and server rows; Samsung Galaxy Tab S10 Lite (Exynos 1380, 8 CPU cores: 4 × Cortex-A78 + 4 × Cortex-A55) for Android (22x on 1 thread).
transformers.js
npm i @huggingface/transformers
npm pkg set type=module
# our text-prep helper: the js/ and frontend/ folders of this repo
pip install -U huggingface_hub
hf download KaleComputer/kale-tts-1-nano --local-dir . --include "js/*" --include "frontend/*"
Save as tts.mjs, then run node tts.mjs:
import { writeFileSync } from "node:fs";
import { StyleTextToSpeech2Model, Tensor } from "@huggingface/transformers";
import { KaleTTS, toWav } from "./js/kale-tts.mjs";
const model = await StyleTextToSpeech2Model.from_pretrained("KaleComputer/kale-tts-1-nano", { subfolder: "", dtype: "fp32" });
// text -> the model's inputs, sentence by sentence
const prep = await KaleTTS.prepare("https://huggingface.co/KaleComputer/kale-tts-1-nano/resolve/main/");
const parts = [];
for await (const { inputs } of prep.inputs("Hello there! It is 9:30 on March 3.", { voice: "kenzie" })) {
const feeds = {};
for (const [name, t] of Object.entries(inputs)) feeds[name] = new Tensor(t.type, t.data, t.dims);
// waveform.data: Float32Array, 24 kHz
const { waveform } = await model(feeds);
parts.push(waveform.data);
}
const audio = new Float32Array(parts.reduce((n, p) => n + p.length, 0));
let at = 0;
for (const p of parts) { audio.set(p, at); at += p.length; }
writeFileSync("hello.wav", Buffer.from(toWav(audio, 24000)));
KaleTTS.prepare() is our small text-prep helper: it turns text into this model's inputs (numbers and dates as words, the checked
dictionary, the voice's style and voice inputs, and for tiny the Ettin word inputs). transformers.js has no tokenizer for these inputs,
so the helper builds them and transformers.js runs the model.
On WebGPU (transformers.js 4.3 or newer), load the float32 file and keep the six small LSTM layers on the CPU for full speed, the same settings our runtime uses:
const LSTM_NODES = ["lstms.0", "lstms.2", "lstms.4", "lstm", "shared", "lstm_1"].flatMap((n) => [`/g/${n}/LSTM`, `/${n}/LSTM`]);
const model = await StyleTextToSpeech2Model.from_pretrained("KaleComputer/kale-tts-1-nano", {
subfolder: "",
dtype: "fp32",
model_file_name: "model.fp32",
device: "webgpu",
session_options: { executionProviders: [{ name: "webgpu", forceCpuNodeNames: LSTM_NODES }] },
});
This is temporary.
LSTM_NODES and session_options) keep six small LSTM layers on the CPU. They stay until an
onnxruntime release with the LSTM fix ships in Transformers.js; then you can remove them.Without transformers.js: our runtime
Download the model with our runtime. It turns any text into speech, numbers, dates and tech terms included:
# the hf command comes with huggingface_hub
pip install -U huggingface_hub
# only the files the runtime needs
hf download KaleComputer/kale-tts-1-nano --local-dir kale-tts-1-nano \
--include "kale-tts-1-nano.onnx" --include "*tokenizer.json" --include "voices.json" --include "config.json" \
--include "voices/*" --include "js/*" --include "frontend/*" --include "python/*" --include "server/*"
cd kale-tts-1-nano
Node.js
npm i onnxruntime-node
npm pkg set type=module
Save as hello.mjs, then run node hello.mjs:
import { writeFileSync } from "node:fs";
import { KaleTTS, toWav } from "./js/kale-tts.mjs";
const tts = await KaleTTS.load("./");
// audio: Float32Array, 24 kHz
const { audio } = await tts.speak("Hello there! It is 9:30 on March 3.", { voice: "kenzie" });
writeFileSync("hello.wav", Buffer.from(toWav(audio, 24000)));
// for long text, sentence by sentence:
// for await (const { audio } of tts.stream(longText, { voice: "amy" })) play(audio);
Python
pip install onnxruntime numpy quickjs soundfile
Save as hello.py in the same folder, then run python hello.py:
import sys
sys.path.insert(0, "python")
import soundfile as sf
from kale_tts import KaleTTS
tts = KaleTTS(".")
# float32 audio, 24 kHz
audio = tts.speak("Hello there!", voice="amy")
sf.write("hello.wav", audio, 24000)
HTTP server
Run it locally and call it from any language or app over HTTP (an OpenAI-shaped POST /v1/audio/speech, WAV or streamed PCM):
npm i onnxruntime-node
# without a token it only listens on 127.0.0.1
TTS_API_TOKEN=choose-a-secret-of-20-characters PORT=8000 node server/server.mjs
curl -s http://127.0.0.1:8000/v1/audio/speech \
-H "Authorization: Bearer choose-a-secret-of-20-characters" \
-d '{"model": "kale-tts-1-nano", "input": "Hello there!", "voice": "kenzie"}' \
-o hello.wav
You pass the voice by its name.
Research agents ran the experiments, on Claude Opus 5.5 and Claude Fable 5.2. We told them what to find out, and an automation checked every 60 minutes whether any agent was stuck and got it moving again.
What we tried, one line each:
Teacher and data
Reading the text, planning, the dictionary
Making the sound
Export
Earlier directions: voices and data
Earlier directions: Matcha-TTS fine-tuning
Earlier directions: other models as the on-device voice
Earlier directions: prosody, speed and serving
Earlier directions: yardsticks and other languages
forceCpuNodeNames; our JS API does it by default): about 1.7x faster on onnxruntime-web 1.27 and newer (83x -> 144x through transformers.js, 18 Nov).kale-tts-1-nano.onnx on CPU and WASM, kale-tts-1-nano.fp32.onnx on WebGPU.kale-tts-1-nano.onnx (14.77 MB: int8 sound generator, float32 for the whole timing path), kale-tts-1-nano.fp32.onnx (30.59 MB, every weight float32, for WebGPU), voices/ + voices.json (one style table and one sound row per voice), tokenizer.json. Inputs: input_ids, style, speed, then the voice inputs listed in voices.json. PyTorch weights in pytorch/ (see pytorch/README.md); the training recipe is in kale-tts-1-tiny.
Weights and code Apache-2.0; dataset CC BY 4.0.
AI-generated speech: everything this model says is synthetic speech made by an AI model, in voices that belong to no real person. Tell listeners it is AI-generated, never present it as a human recording, and never use it to imitate a real person.
Credits: based on Maya-1: all training audio is original speech from Maya-1 (Maya Research, Apache-2.0), decoded with SNAC (MIT). The teacher is Kokoro-82M (hexgrad, Apache-2.0), fine-tuned with StyleTTS2 (MIT) and kikiri-tts (Apache-2.0) code; kale-tts-1-tiny keeps its timing-and-melody planner and contains the Ettin 17M encoder (JHU CLSP, MIT). Pronunciations: CMUdict (BSD-2-Clause), Britfone (MIT); the number, date and unit rules come from dizione and the homograph tables from tiny-voca, both by n4ze3m (Apache-2.0), the tables trained on Google Research's Wikipedia Homograph Data (Apache-2.0). Training texts: public-domain books made available by Project Gutenberg. Dataset: kale-tts-1-data (CC BY 4.0).
pytorch/teacher/kale-tts-1-teacher.pth 327 MBpytorch/nano/earlier/sound_side_t10r.pt 34 MBpytorch/nano/sound_side_flrd.pt 34 MBkale-tts-1-nano.fp32.onnx 31 MBkale-tts-1-nano.onnx 15 MBpytorch/nano/student_text_side_flt.pt 12 MBpytorch/nano/earlier/student_text_side_t10t2.pt 12 MBfrontend/words/g2p.bin 2 MBfrontend/words/en-gb/g2p.bin 2 MBpytorch/teacher/voices/h013f.pt 523.7 kBpytorch/teacher/voices/s057b.pt 523.7 kBpytorch/teacher/voices/t071.pt 523.7 kBpytorch/teacher/voices/w165.pt 523.7 kBpytorch/teacher/voices/w602.pt 523.7 kBpytorch/teacher/voices/x328.pt 523.7 kBpytorch/teacher/voices/d03.pt 523.7 kBpytorch/teacher/voices/d13.pt 523.7 kBpytorch/teacher/voices/f06.pt 523.7 kBpytorch/teacher/voices/m3.pt 523.7 kBvoices/d03.bin 522.2 kBvoices/d13.bin 522.2 kBvoices/f06.bin 522.2 kBvoices/h013f.bin 522.2 kBvoices/m3.bin 522.2 kBvoices/s057b.bin 522.2 kBvoices/t071.bin 522.2 kBvoices/w165.bin 522.2 kBvoices/w602.bin 522.2 kBvoices/x328.bin 522.2 kBvoices/d03.extra.bin 0.3 kBvoices/d13.extra.bin 0.3 kBvoices/f06.extra.bin 0.3 kBvoices/h013f.extra.bin 0.3 kBvoices/m3.extra.bin 0.3 kBvoices/s057b.extra.bin 0.3 kBvoices/t071.extra.bin 0.3 kBvoices/w165.extra.bin 0.3 kBvoices/w602.extra.bin 0.3 kBvoices/x328.extra.bin 0.3 kBKaleComputer/kale-tts-1-base