LoRA adapters and TensorBoard logs for Hypa-Orpheus 3B TTS/VC.
This repository is the LoRA adapter companion to hypaai/Hypa-Orpheus-3b-TTS-VC.
Both models belong to the Hypa-Orpheus collection and describe the same Step 3a production fine-tune. The only difference is what is stored here:
| Repository | Contents | Best for |
|---|---|---|
hypaai/Hypa-Orpheus-3b-TTS-VC |
Merged 16-bit weights | Inference, vLLM serving, production deployment |
hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs (this repo) |
LoRA adapters + TensorBoard logs | Continued fine-tuning, adapter inspection, training metrics |
For the full model description, capabilities, prompt format, languages, training data, hyperparameters, evaluation, limitations, citation, and acknowledgements, see the main README:
hypaai/Hypa-Orpheus-3b-TTS-VC README
Adapter at a Glance
| Item | Value |
|---|---|
| Base model | unsloth/orpheus-3b-0.1-ft |
| Method | 4-bit QLoRA via Unsloth |
| LoRA rank / alpha | r=512, α=512 |
| Trainable parameters | 778M / 4.08B (19.07%) |
| Shipped checkpoint | Step 44,334 (validation-best) |
| Best validation loss | 3.513 @ step 44,334 |
| License | Apache 2.0 |
TensorBoard
Interactive training logs for the production run:
View TensorBoard on Hugging Face
Headline metrics from the shipped run:
| Metric | Value | Step |
|---|---|---|
| Initial training loss | 4.743 | 1 |
| Best training loss | 3.549 | 26,604 |
| Final training loss | 3.560 | 42,123 |
| Initial validation loss | 3.870 | 2,217 |
| Best validation loss (shipped) | 3.513 | 44,334 |
Validation loss improved at every evaluation interval. The merged release in the sibling repository uses this validation-best checkpoint.
Quick Usage
Load adapters for continued fine-tuning (Unsloth)
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/orpheus-3b-0.1-ft",
max_seq_length=2048,
dtype=None,
load_in_4bit=True,
)
model = FastLanguageModel.get_peft_model(
model,
r=512,
lora_alpha=512,
target_modules=[
"q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj",
],
lora_dropout=0.0,
bias="none",
use_gradient_checkpointing="unsloth",
)
# Load the shipped Hypa-Orpheus adapter weights
from peft import PeftModel
model = PeftModel.from_pretrained(model, "hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs")
Merge to 16-bit for inference
After loading the adapter, merge into full weights for deployment (same artifact as the sibling repo):
model = model.merge_and_unload()
model.save_pretrained("Hypa-Orpheus-3b-TTS-VC-merged")
tokenizer.save_pretrained("Hypa-Orpheus-3b-TTS-VC-merged")
For production inference, you can also use the pre-merged checkpoint directly:
hypaai/Hypa-Orpheus-3b-TTS-VC
Repository Contents
Hypa-Orpheus-3b-TTS-VC-LoRAs/
├── README.md
├── adapter_config.json
├── adapter_model.safetensors
├── training_args.bin
└── runs/
└── events.out.tfevents.* # TensorBoard scalars
Related Resources
- Merged model: hypaai/Hypa-Orpheus-3b-TTS-VC
- Training data (public subset): Hypa-Voices collection
- Dataset analysis: Hypa-Voices statistical breakdown
- Upstream base: unsloth/orpheus-3b-0.1-ft
For questions or contributions, contact chris@hypaintelligence.com or open an issue on Hugging Face.