voicetut-egyptian-emotion

VoiceTut-TTS (OmniVoice 0.6B, Egyptian Arabic) fine-tuned on masterofaudio2077/egyptian-emotion-tts: 19,480 Egyptian Arabic clips (46.4 hours) in one voice and 20 emotions. Put an emotion tag in front of the text and the model reads the line in that emotion.

Tutorial: One Tag, Any Emotion: Egyptian Arabic TTS That Actually Feels, the full recipe step by step on Kaggle's free T4s.

Listen

Morning with coffee

#	Emotion	Text
1	tired	صباح الخير، انا لسه مش قادر افتح عيني.
2	calm + sighs	اه، ريحة القهوة دي بتصحي الواحد.
3	happy	اول رشفة كده والدنيا اتحسنت خالص.
4	surprised	استنى! هو الساعة بقت تسعة؟
5	urgent	يا نهار ابيض، الاجتماع بعد نص ساعة، لازم اجري.

Shopping with friends

#	Emotion	Text
1	excited	يلا يا جماعة، النهارده هنقلب المول كله!
2	playful	دي الجاكيت اللي كنتي بتحلمي بيها، جربيها بسرعة.
3	surprised + gasps	ايه ده؟ سعرها تلات تلاف جنيه؟
4	sarcastic	لا طبعا، ده انا ممكن اشتري بيه عربية.
5	tender	مبسوطة اوي اننا خرجنا سوا، وحشتوني.

Joking with friends

#	Emotion	Text
1	playful + chuckles	يا عم انت كل مرة تقول هعمل دايت وتخلص طبق الكشري لوحدك.
2	sarcastic	اه طبعا، انت اللي هتكسب الماتش، ده انت بتنهج من السلم.
3	excited	استنوا استنوا، لسه الحتة الحلوة في الحكاية جاية!
4	happy + laughs	خلاص والله ضحكتوني، بطني وجعتني من الضحك.
5	shouting	يا جماعة ركزوا بقى، الاكل هيبرد!

Remembering someone who passed

#	Emotion	Text
1	sad	الله يرحمها، كانت اطيب واحدة في العيلة كلها.
2	tender	فاكرين لما كانت بتستنانا على الباب كل جمعة؟
3	sad + sighs	مكنتش متخيل ان البيت هيبقى فاضي كده من غيرها.
4	calm	ادعولها كتير، هي محتاجة دعانا دلوقتي.
5	whisper	ربنا يصبرنا ويجمعنا بيها في الجنة.

Usage

Install OmniVoice from github.com/k2-fsa/OmniVoice, then:

import json, torch, soundfile as sf
from huggingface_hub import snapshot_download
from omnivoice import OmniVoice
path = snapshot_download("masterofaudio2077/voicetut-egyptian-emotion")
import sys; sys.path.insert(0, path)
from omni_emotion import generate, seconds

model = OmniVoice.from_pretrained(path, device_map="cuda:0", dtype=torch.float16)
rates = json.load(open(f"{path}/rates.json"))
text = "[with warm, genuine happiness] اول رشفة كده والدنيا اتحسنت خالص."
audio = generate(model, text, seconds(text, "happy", rates))
sf.write("out.wav", audio, 24000)

generate sets the length from the emotion's speaking rate (rates.json) and keeps the output level, so whispers stay quiet and shouts stay loud. OmniVoice's own model.generate counts the English tag as speech and often reads it aloud.

Results

8 held-out sentences, each read in all 20 emotions.

	Dataset	VoiceTut-TTS	This model
Character error rate (Whisper large-v3)		0.192	0.025
Speaker similarity (WavLM)		0.539	0.960
Loudness range across emotions (dB)	7.8	2.9	4.7
Pitch range across emotions (Hz)	87	23	109
Emotion loudness order vs dataset (correlation)		0.43	0.82
Emotion pitch order vs dataset (correlation)		0.23	0.83

Training

Full fine-tune of VoiceTut-TTS, 7500 steps on 2x T4, Lion optimizer (lr 3e-6, weight decay 0.03), fp16. This is step 7500.
18,980 training clips; 500 sentences held out.
Audio tokenised at each clip's own level (OmniVoice's default normalises it, which erases loudness).
No reference clip during training and mask ratio 0.9-1.0, so the tag alone carries the emotion.

Limitations

One voice only. Cloning another voice from a reference clip is not supported.
The training audio is synthetic (Fish Audio Drama 3), so the emotions follow Drama 3's style.
Not for impersonating a real person or deceiving listeners.

Acknowledgements

VoiceTut-TTS, OmniVoice by k2-fsa, the Higgs audio tokenizer v2, and Fish Audio Drama 3 for generating the training audio.

masterofaudio2077/voicetut-egyptian-emotion