hypaai/Hypa-Orpheus-3b-TTS-VC-ext

Extended continued fine-tune of Hypa-Orpheus 3B with a higher-capacity LoRA adapter (r=2048), merged to 16-bit bf16 for inference.

Model Description

Hypa-Orpheus 3B Extended (hypaai/Hypa-Orpheus-3b-TTS-VC-ext) is a continued fine-tune of the production Hypa-Orpheus 3B release. It uses the same Hypa-Voices corpus, prompt formats, and SNAC training pipeline as the primary model, but attaches a much higher-rank LoRA adapter (r=2048, α=2048) on top of the merged Step 3a checkpoint and trains further on the same precomputed integer SNAC shards.

This is an experimental capacity extension, not a replacement for the primary release. It exists for researchers and deployers who want the strongest held-out loss we observed after continuing training with a larger adapter.

Collection: hypaai/hypa-orpheus

Capabilities, languages, prompt structure, SNAC decode path, and intended use cases are identical to the primary model. See the main model card for the full description:

hypaai/Hypa-Orpheus-3b-TTS-VC README

---

What Changed in This Run

The extended run kept the same data mixture and response-only SNAC supervision, but increased adapter capacity and batch size while continuing from the already merged production weights:

The published extended checkpoint reflects 4,992 optimizer steps, about 30% of one effective epoch on the same interleaved TTS/VC mixture. Training was stopped and released at the validation-best step observed in that continuation segment.

---

Evaluation and Checkpoint Selection

TensorBoard logs for the extended adapter:

Hypa-Orpheus 3B Extended LoRA training logs

Headline metrics from the shipped continuation:

Compared with the primary release validation loss of 3.513 at step 44,334, the extended checkpoint improves held-out loss by 0.075 (approximate validation perplexity 33.5 → 31.1). Validation loss improved at every logged evaluation interval in this continuation.

As with the primary release, these scalars measure uncertainty over the supervised target SNAC stream. They do not, by themselves, prove better perceptual quality. The extended checkpoint should be treated as a loss-improved continuation pending formal listening and task benchmarks.

Gradient norms remained stable throughout the run (roughly 0.18 → 0.11).

---

Which Checkpoint Should I Use?

Choose the primary release when you want the fully budgeted Step 3a training run and the smaller r=512 adapter footprint. Choose this extended release when you want the best validation loss we published after continuing from that checkpoint with a r=2048 adapter, and can accept the larger merged weights and shorter continuation schedule.

---

Quick Start

Load the extended merged model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "hypaai/Hypa-Orpheus-3b-TTS-VC-ext"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

Prompt assembly, SNAC encode/decode, and vLLM serving follow the same conventions as the primary release. This repository includes the same handler.py inference entrypoint pattern as hypaai/Hypa-Orpheus-3b-TTS-VC.

vllm serve hypaai/Hypa-Orpheus-3b-TTS-VC-ext --dtype bfloat16 --max-model-len 4096

Load the extended LoRA adapter

See hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext for adapter loading, merging, and TensorBoard links.

---

Limitations (Extended Release)

• Shorter continuation budget. The shipped extended checkpoint covers 4,992 steps, not the full planned 16,626-step schedule.
• Much larger adapter. r=2048 increases adapter storage and merge size roughly 4× versus the primary release.
• Experimental status. The primary release remains our default production artifact; this continuation is published for transparency and research comparison.
• Same perceptual eval gap. Improved held-out loss does not yet come with published WER/CER, speaker-similarity, or MOS results.

For shared limitations on languages, prompt sensitivity, and context length, see the primary model card.

---

Released Artifacts

• Extended merged 16-bit model (this repo): hypaai/Hypa-Orpheus-3b-TTS-VC-ext
• Extended LoRA + TensorBoard: hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs-ext
• Primary merged release: hypaai/Hypa-Orpheus-3b-TTS-VC
• Primary LoRA + TensorBoard: hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs
• Training data (public subset): Hypa-Voices collection

For questions or contributions, contact chris@hypaintelligence.com or open an issue on Hugging Face.

---

Citation

If you use the extended Hypa-Orpheus checkpoint, please cite both the primary release and this continuation:

@misc{hypaorpheus2026,
  title        = {Hypa-Orpheus 3B: Multilingual Text-to-Speech and Voice Cloning for Underrepresented Languages},
  author       = {{Hypa Intelligence and AfroVoices}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC}}
}

@misc{hypaorpheus2026ext,
  title        = {Hypa-Orpheus 3B Extended: Continued Fine-Tuning with Higher-Capacity LoRA},
  author       = {{Hypa Intelligence and AfroVoices}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC-ext}}
}

---

License

Released under the Apache License 2.0.

---

Acknowledgments

Same upstream acknowledgments as the primary Hypa-Orpheus release: Canopy Labs (Orpheus), Meta AI (Llama), Hubert Siuzdak (SNAC), Unsloth, RunPod, Mozilla Common Voice, AfroVoices, and Hypa Intelligence Research (HaIR).

---

Hypa Intelligence • Website • Hugging Face • Updates