Russian Emotional TTS (350M DiT / Flow Matching)

An advanced Russian Text-to-Speech (TTS) and voice cloning model based on the DiT (Diffusion Transformer) / Flow-Matching architecture (F5-TTS). Fine-tuned on high-fidelity, expressive theatrical dialogues with lexical stress annotation.

🚀 Quick Online Demo (Google Colab Free GPU)

1. Download the notebook: russian_emotional_tts_colab.ipynb
2. Go to Google Colab -> Click Upload -> select the downloaded file.
3. Switch runtime to T4 GPU (Runtime -> Change runtime type -> T4 GPU) and run all cells.
4. Open the generated gradio.live link to use the Web UI.

Key Highlights

• Parameters: ~350M (337M DiT acoustic backbone + 13.5M Vocos 24kHz neural vocoder).
• Sample Rate: 24,000 Hz studio mono audio.
• Lexical Stress Support: Uses explicit + markers before stressed vowels (e.g., прив+ет, к+ак дел+а?) for natural pronunciation and homograph disambiguation.
• Zero-Shot Voice Cloning: High-fidelity timbre cloning using a 3–10 second clean reference audio clip.
• Format: Pure .safetensors (zero pickle risks, instantaneous tensor loading).

---

Quickstart (Inference)

1. Installation

pip install f5-tts soundfile torchaudio

2. Python Code

import torch
import soundfile as sf
from f5_tts.infer.utils_infer import infer_process, load_model, load_vocoder
from f5_tts.model import DiT

device = "cuda" if torch.cuda.is_available() else "cpu"

# Load neural vocoder
vocoder = load_vocoder(vocoder_name="vocos", device=device)

# Load acoustic DiT model
model_cfg = dict(dim=1024, depth=22, heads=16, ff_mult=2, text_dim=512, conv_layers=4)
model = load_model(
    DiT,
    model_cfg,
    "model.safetensors",
    vocab_file="vocab.txt",
    device=device
)

# Zero-shot inference
wave, sr, _ = infer_process(
    ref_audio="reference.wav",
    ref_text="Текст из образца голоса",
    gen_text="Прив+ет! Это русская нейрос+еть для озв+учки.",
    model_obj=model,
    vocoder=vocoder,
    device=device,
    nfe_step=32,
    speed=1.0
)

sf.write("output.wav", wave, sr)

---

Training Details

• Base Architecture: F5-TTS DiT Base v2
• Fine-tuning Dataset: 14+ hours of studio-grade emotional Russian actor conversations (langswap/dialogs-ru-emotional-conversations).
• Hardware: Single NVIDIA GeForce RTX 5060 (8GB VRAM).
• Optimizer: 8-bit Adam (bitsandbytes) with dynamic batching.
• License: MIT (Free for commercial and personal use).