speecht5_finetuned_sindhi_digits

A fine-tuned version of microsoft/speecht5_tts for Sindhi spoken digit synthesis (0–9)

• Eval Loss: 0.5491
• Dataset: DanishMahdi/sindhi-spoken-digits-tts
• Speakers: Multiple speakers conditioned via x-vectors
• Audio: 16kHz mono

How to Use

import torch
import numpy as np
import soundfile as sf
from IPython.display import Audio as IPythonAudio
from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor

# ── Load model ──────────────────────────────────────────────────────────────
model_id  = "DanishMahdi/speecht5_finetuned_sindhi_digits"
processor = SpeechT5Processor.from_pretrained(model_id)
model     = SpeechT5ForTextToSpeech.from_pretrained(model_id)
vocoder   = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")

# ── Speaker embedding ───────────────────────────────────────────────────────
# speaker_mean_embeddings: dict of {speaker_id: 512-dim numpy array}
# obtained by averaging x-vectors across all utterances of each speaker
target_speaker = list(speaker_mean_embeddings.keys())[0]
speaker_emb = torch.tensor(
    np.array(speaker_mean_embeddings[target_speaker], dtype=np.float32)
).unsqueeze(0)  # shape: (1, 512)

# ── Normalize Sindhi script
text = normalize_text("ھڪ")          #  (1)
print(f"Input: 'ھڪ' → normalized: '{text}'")

# ── Generate speech ─────────────────────────────────────────────────────────
inputs = processor(text=text, return_tensors="pt")
speech  = model.generate_speech(inputs["input_ids"], speaker_emb, vocoder=vocoder)

display(IPythonAudio(speech.numpy(), rate=16000))
sf.write("output.wav", speech.numpy(), samplerate=16000)

Training results

Limitations

• Digits 0–9 only — does not generalize to arbitrary Sindhi text
• Best quality for speakers seen during training

Citation

@inproceedings{ao-etal-2022-speecht5,
    title     = {SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing},
    author    = {Ao, Junyi and Wang, Rui and Zhou, Long and others},
    booktitle = {Proceedings of ACL 2022},
    year      = {2022}
}

@misc{mahdi-2026-sindhi-digits-tts,
    title     = {SpeechT5 Fine-tuned for Sindhi Spoken Digit Synthesis},
    author    = {Danish Mahdi},
    year      = {2026},
    publisher = {Hugging Face},
    url       = {https://huggingface.co/DanishMahdi/speecht5_finetuned_sindhi_digits}
}