adamw-PT
494M Qwen2.5-0.5B-shaped model (tied embeddings, 4,096 context), pretrained from scratch on 200B DCLM-Baseline tokens (95,368 steps, batch 512 x 4,096). Final held-out loss: DCLM 2.8364, FineMath 2.1915 (full validation sets). Part of a reproduction of Data Schedules and Learning Rate Schedules Should Be Designed Together (arXiv 2610.29348) with Megatron-LM; converted to Hugging Face format with Megatron-Bridge.