Music Llama

Llama for MIDI music generation

---

Model Details

Model Description

• Developed by: SkyTNT
• Trained by: asigalov61
• Model type: Llama Transformer
• License: apache-2.0

Model Sources

• Code Repository: https://github.com/SkyTNT/midi-model
• Demo: https://huggingface.co/spaces/skytnt/midi-composer

Training Details

Training Data

• projectlosangeles/Monster-MIDI-Dataset

Training Hyperparameters

• config: tv2o-large
• max-len: 4096
• layers: 24
• acc-grad: 8
• lr: 1e-4
• weight-decay: 0.01
• batch: 8x2x8
• fp16 mixed precision

Loss

• train loss: 0.32

---

Project Los Angeles

Tegridy Code 2024