MetaCog-V2-300M

MetaCog-V2 is a 301,268,512-parameter dialogue language model trained from scratch with Meta V2 explicit recurrent state: four levels, eight memory slots per level, learned read/write/erase, confidence and routing. The released inference export contains the SFT checkpoint at step 1526 (500,039,680 tokens).

Training

• Pretraining: 6,500,188,160 tokens
• SFT: 500,039,680 tokens
• SFT carry validation: loss 1.6080, PPL 4.993
• SFT reset validation: loss 1.6283, PPL 5.095

Evaluation

benchmarks/meta_sft_bench.json contains the exact run. Knowledge subset (50 examples/task): ARC-Easy 40.00%, HellaSwag 26.00%, Winogrande 52.00%, BoolQ 22.00%. The ordinary-attention control in the speed table is untrained; therefore no quality comparison to that control is claimed. Speed measurements use the same GPU and sequence length.

Loading

Install liger-kernel and load with trust_remote_code=True. Persistent state is available through forward_hidden(..., state=..., reset_mask=...); see chat.py.

Limitations

This is a research checkpoint. The short generation smoke test produced immediate stop outputs on the tested prompts; treat dialogue quality as experimental, not production-ready.