We introduce LLaDA, a diffusion model with an unprecedented 8B scale, trained entirely from scratch, rivaling LLaMA3 8B in performance.
[2025-10-21] We have modified modeling_llada.py to support the input of attention_mask.
model-00003-of-00006.safetensors 3.0 GBmodel-00004-of-00006.safetensors 3.0 GBmodel-00002-of-00006.safetensors 2.9 GBmodel-00005-of-00006.safetensors 2.9 GBmodel-00006-of-00006.safetensors 2.1 GBmodel-00001-of-00006.safetensors 2.1 GB