First attempt at a rank-128 attention-only LoRA on the suno-reggae split. During the run the model seemed to barely move and the working theory was that the learning rate was simply too low. The real problem turned out to be the validation audio duration: clips were rendered far too short (30 seconds) to expose what training was doing, and this was only realised after v2 had already been trained. Once the validation duration was increased, the damage to musical structure was evident — see the v2 card for the root cause (misaligned LM-rollout conditioning without an RVQ encoder).
LoRA strength is not a useful mitigation here: the adapter does effectively nothing up to ~0.8 strength, then quickly degrades the output approaching 1.0, at which point it is firmly in "experimental music" territory.
This is a PEFT LoRA derived from MiniMaxAI/MiniMax-Music3.
The main validation prompt used during training was:
laid-back roots reggae with warm bass, skanking guitar, dub delays and soulful vocals
1.70.030FlowMatchEulerDiscreteScheduler42256Note: The validation settings are not necessarily the same as the training settings.
The text encoder was not trained. You may reuse the base model text encoder for inference.
Training epochs: 32
Training steps: 4000
Learning rate: 8e-05
Max grad value: 1.0
Effective batch size: 8
Gradient checkpointing: True
Prediction type: flow_matching
Optimizer: adamw_bf16
Trainable parameter precision: Pure BF16
Base model precision: no_change
Caption dropout probability: 0.1%
LoRA Rank: 128
LoRA Alpha: 128.0
LoRA Dropout: 0.1
LoRA initialisation style: default
LoRA mode: Standard
import torch
from diffusers import DiffusionPipeline
model_id = 'MiniMaxAI/MiniMax-Music3'
adapter_id = 'bghira/minimax-music-suno-reggae-rank128'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)
prompt = "laid-back roots reggae with warm bass, skanking guitar, dub delays and soulful vocals"
negative_prompt = 'blurry, cropped, ugly'
## Optional: quantise the model to save on vram.
## Note: The model was not quantised during training, so it is not necessary to quantise it during inference time.
#from optimum.quanto import quantize, freeze, qint8
#quantize(pipeline.transformer, weights=qint8)
#freeze(pipeline.transformer)
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
prompt=prompt,
negative_prompt=negative_prompt,
num_inference_steps=30,
generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
width=256,
height=256,
guidance_scale=1.7,
).images[0]
model_output.save("output.png", format="PNG")
30-second renders of the fixed validation prompt/lyrics at training milestones (full set in the repo's assets directory):
pytorch_lora_weights.safetensors 151 MBcheckpoint-1000/pytorch_lora_weights.safetensors 151 MBcheckpoint-1250/pytorch_lora_weights.safetensors 151 MBcheckpoint-1500/pytorch_lora_weights.safetensors 151 MBcheckpoint-1750/pytorch_lora_weights.safetensors 151 MBcheckpoint-2000/pytorch_lora_weights.safetensors 151 MBcheckpoint-2250/pytorch_lora_weights.safetensors 151 MBcheckpoint-250/pytorch_lora_weights.safetensors 151 MBcheckpoint-2500/pytorch_lora_weights.safetensors 151 MBcheckpoint-2750/pytorch_lora_weights.safetensors 151 MBcheckpoint-3000/pytorch_lora_weights.safetensors 151 MBcheckpoint-3250/pytorch_lora_weights.safetensors 151 MBcheckpoint-3500/pytorch_lora_weights.safetensors 151 MBcheckpoint-3750/pytorch_lora_weights.safetensors 151 MBcheckpoint-4000/pytorch_lora_weights.safetensors 151 MBcheckpoint-500/pytorch_lora_weights.safetensors 151 MBcheckpoint-750/pytorch_lora_weights.safetensors 151 MB有文件?提交种子。
提交种子MiniMaxAI/MiniMax-Music3