AuroraTTS-100K

AuroraTTS-100K is a very small English TTS model with 95,638 inference parameters. It was trained from scratch on LJSpeech, without distillation, and targets the sub-100K parameter regime.

The model has no neural vocoder. The network predicts per-frame synthesis parameters (normalized log-F0, voicing, a 40-band harmonic spectral envelope and a 24-band noise spectral envelope), and a differentiable harmonic+noise DSP synthesizer renders the waveform. The DSP block has only 145 learned parameters and is included in the total inference parameter count.

---

Usage

import soundfile as sf
from transformers import AutoModel

# variant: omit for FP32 (374 KB), "fp16" (187 KB), or "int8" (111 KB)
model = AutoModel.from_pretrained(
    "AuroraAI-Research/AuroraTTS-100K",
    trust_remote_code=True,
    variant="fp16",
).eval()

wav, sr = model.synthesize("The quick brown fox jumps over the lazy dog.")
sf.write("out.wav", wav, sr)

Requirements: PyTorch, torchaudio, NumPy, soundfile, g2p_en, and nltk. The text frontend downloads the required NLTK data on first use. Batch size is 1.

Optional CPU ONNX inference without PyTorch

The ONNX companion provides the existing FP32 encoder and decoder exports, a NumPy/SciPy harmonic-noise waveform renderer, the exporter, and reproducible validation evidence. The original Transformers usage above is unchanged. From the onnx directory, follow README.txt to synthesize with ONNX Runtime and the original g2p_en/NLTK pronunciation frontend; PyTorch and Transformers are needed only to reproduce the export/reference comparison.

The optional compact native frontend is supplied for review as source only. Its six pronunciation-data files and a prebuilt DLL are not included because STEER's broader frontend-data licence wording requires clarification. The g2p_en/NLTK route does not require those STEER assets. Component licences remain separate: Aurora's original model and derived graphs are Apache-2.0; the supplied Sonata tools/bridge are GPL-3.0-or-later; STEER C source retains its MIT notices. See the companion's LICENSES and README for details and measured limitations.

---

Evaluation

Held-out set: 24 sentences from the SanoTTS English diverse held-out set. UTMOS: SpeechMOS utmos22_strong, reported as mean ± std over sentences.

AuroraTTS-100K

For the INT rows, weights are rounded to the target grid and dequantized to fp32 for compute. This measures rounding damage, not the speed of a real integer kernel. Biases, norms, and DSP parameters remain fp32.

External baselines

Speed

Real-time factor (RTF), lower is faster. FP32.

---

Architecture

AuroraTTS-100K is a non-autoregressive TTS model with learned alignment and a parametric waveform synthesizer. The design is close to FastPitch / RAD-TTS, but replaces the neural vocoder with a tiny differentiable DSP synthesizer.

During training, durations and token-level pitch are teacher-forced from the alignment. At inference, the duration and pitch predictors are used directly.

DSP synthesizer

• Up to 80 harmonics, with amplitudes sampled from the predicted 40-band mel-spaced envelope.
• Noise is shaped by the predicted 24-band envelope in the STFT domain (1024-point FFT, hop 256).
• Learned per-harmonic phase offsets (80 parameters).
• Learned static output FIR filter (65 taps).
• Phase is accumulated in float64 during both training and inference to avoid drift over long utterances.

---

Training methodology

Text frontend

• English text is converted to phonemes with g2p_en.
• Vowels carry ARPAbet stress markers.
• The vocabulary has 79 symbols: padding, silence, phonemes, punctuation, and space.
• Repeated spaces are collapsed, and silence tokens are added at utterance boundaries.

Data preparation

• Dataset: LJSpeech-1.1, 22.05 kHz, 13,100 clips. A fixed split (seed 1234) gives 128 test clips, 64 validation clips, and the remaining 12,908 for training.
• Clips longer than 12 s are excluded, as are clips with fewer than 5 voiced frames or too few frames for their transcript.
• F0 is extracted with PyWorld (harvest by default, dio optional) and refined with stonemask, with a floor of 70 Hz and a ceiling of 450 Hz.
• Unvoiced frames are filled by interpolating log-F0 from voiced frames for training targets.
• Global F0 mean and standard deviation (log-Hz) are computed from voiced training frames and used for normalization.

Alignment

Alignment is learned during training and is not used at inference.

• A train-time text/mel aligner produces frame-token alignment scores from 80-band log-mels.
• A beta-binomial prior encourages monotonic, near-diagonal attention. It is computed on the GPU with lgamma.
• Forward-sum loss enforces monotonic alignment.
• Monotonic alignment search (MAS), compiled with numba, extracts hard token durations.
• The first 3,000 steps train only the aligner. The binary alignment loss then ramps in linearly over the following 10,000 steps.

Losses

The acoustic model is trained with:

• duration loss (MSE in log space)
• token-level pitch loss
• frame-level F0 L1 loss, weighted toward voiced frames
• voiced/unvoiced binary cross-entropy
• multi-resolution STFT loss (FFT sizes 256 to 2048)
• log-mel L1 loss

Waveform losses are computed on random 128-frame crops (about 1.5 s), using teacher-forced F0 and voicing passed through the DSP synthesizer. Loss weights: STFT 1.0, log-mel 2.0, F0 2.0, voicing 0.5.

From step 90,000, a multi-resolution spectrogram discriminator (FFT sizes 512, 1024, 2048) adds least-squares adversarial and feature-matching losses (weights 0.5 and 2.0). The discriminator is train-time only.

Optimization