hibari

Tiny English text-to-speech. 7.9 MB int8 ONNX models, 6.15 M parameters each, about 130x realtime on a single CPU core, 24 kHz audio. Each voice is distilled from one Kokoro-82M voice: the default (af_heart) scores UTMOS 4.43 against the teacher's 4.52 while running 65x faster. Five voices: heart (American female, inside the pip package), fenrir (American male), emma (British female), fable (British male) and kore (American female), the last four in voices/ here and downloaded by the package on first use. Comes with its own text-to-phoneme front end (no spaCy, no espeak-ng), so plain text goes in and audio comes out.

hibari is Japanese for skylark. Code, Python package and training pipeline: github.com/MMWilliams/hibari.

Samples

hello:

numbers, dates, money:

question:

story:

The other voices introducing themselves (same sentence, synthesized by the package):

fenrir (American male):

emma (British female):

fable (British male):

kore (American female):

Use

pip install hibari-tts
hibari "Hello there, this is hibari." -o hello.wav

from hibari import Hibari
tts = Hibari()                         # voice "heart"; Hibari(voice="emma") or "fenrir" download once
audio = tts("Hello there.")            # numpy float32, 24 kHz
tts.save("Hello there.", "hello.wav")
for chunk in tts.stream(long_text):    # one array per sentence, for low latency
    ...

hibari "Hello there." --voice fenrir -o fenrir.wav
hibari --list-voices

The PyPI name is hibari-tts; the import and the command are hibari. The package depends only on numpy and onnxruntime and runs on Windows, macOS and Linux (x64 and arm64). Everything it needs is inside the wheel; this repository holds the same files for people who want to port or study the model.

Files

ONNX interface: input ids [1, L] int64 (phoneme ids from g2p/vocab.json, id 0 at both ends) and speed [1] float32; outputs lh, theta, ln, lr, phi [1, 601, F] and f0 [1, F], one frame per 12.5 ms. A short numpy routine (pulse train from f0, one inverse STFT) turns them into audio; the formula and reference code are in the GitHub README and src/hibari/dsp.py.

To load a checkpoint in the research code (training/tiny_model.py on GitHub):

import json
from safetensors import safe_open
from safetensors.torch import load_file
from tiny_model import TinyKokoro
p = "checkpoints/decoder_v3_40k.safetensors"
kw = json.loads(safe_open(p, "pt").metadata()["kw"])
m = TinyKokoro(**kw); m.load_state_dict(load_file(p))

Model

The network stops at a parametric spectrum per frame: a harmonic branch (magnitude and phase rotation applied to the STFT of a band-limited pulse train built from F0), a noise branch and a free-phase branch. The host turns it into audio with one inverse STFT, which keeps per-sample work out of the network.

Evaluation

50 sentences never synthesized in training; UTMOS (utmos22_strong, higher is better), word error rate of Whisper base transcripts; speed on one onnxruntime thread, Ryzen 9 9900X.

Per voice (same protocol, 50 unseen sentences each, student fed the teacher's phonemes, one thread):

• emma: snapshot 40k picked by UTMOS with the voice's ProsodyNet on 50 held-out training rows (teacher 4.394; 20k 4.107, 30k 4.125, 40k 4.174, 50k 4.174, 60k 4.173).
• fenrir: snapshot 30k picked by UTMOS with the voice's ProsodyNet on 50 held-out training rows (teacher 4.496; 10k 4.382, 15k 4.400, 20k 4.397, 25k 4.403, 30k 4.414).
• fable: snapshot 25k picked by UTMOS with the voice's ProsodyNet on 50 held-out training rows (teacher 4.333; 10k 4.143, 15k 4.167, 20k 4.151, 25k 4.176, 30k 4.161).
• kore: snapshot 25k picked by UTMOS with the voice's ProsodyNet on 50 held-out training rows (teacher 4.521; 10k 4.454, 15k 4.465, 20k 4.466, 25k 4.471, 30k 4.469).

Text to phonemes, against the reference pipeline (misaki 0.9.4 + spaCy + espeak-ng) on 2,500 held-out sentences: 0.50% phoneme error rate, 88% of sentences identical (95% for sentences with no out-of-vocabulary word). Feeding the same model from hibari's G2P versus the reference G2P on 50 unseen sentences: UTMOS 4.431 vs 4.431, WER 2.6% vs 2.7%.

Intended use and limitations

For reading English text aloud on devices without a GPU, where size and latency matter, and for research on distillation and small vocoders. Not for voice cloning or impersonation, or for use where listeners aren't told the speech is synthetic.

• Five fixed voices (American female x2 and male, British female and male), English only; no speaker or style input. Speed is adjustable.
• Trained on declarative web prose: questions and exclamations have a narrower pitch range than the teacher.
• Sentences are synthesized one at a time, with no cross-sentence prosody.
• Unknown words (about 1 in 5 sentences has one) are read by a small network that gets about 72% of unseen names exactly right; foreign names can come out anglicized.
• Numbers, dates and times follow US conventions.

Training data

All training audio was synthesized by Kokoro-82M from 30 k sentences (66 hours) of agentlans/high-quality-english-sentences. No human recordings were used.

License

Apache-2.0, like Kokoro-82M and misaki. See NOTICE for attributions (Kokoro-82M, kokoro and misaki by hexgrad; spaCy's tokenizer rules, MIT; StyleTTS2, MIT). espeak-ng (GPL) only labeled training words for the out-of-vocabulary net and is not included.