Vega-SupraNeo-4M-Tetris-Verbose

Note: AI was used in the creation of this project. But that's why you're here, isn't it?

---

This is a Tetris finetune of SupraNeo-4M...

A new small finetuning corpus was constructed with data from https://huggingface.co/spaces/DedeProGames/SLM-Tetris-Arena, and this model is intended to dominate. And, oh hell yes, it dominates.

It is HILARIOUSLY good compared to all other models at playing Tetris, and it handily beats every model in the arena, no questions asked.

Benchmarks

Good at Tetris.

Model specs

• Layers: 12
• Hidden size: 160
• Attention heads: 4
• KV heads: 2
• Context length: 1K
• Intermediate size: 432
• Vocabulary: 4,096
• Parameters: 4,070,240
• OG Creator: SupraLabs

Thanks SupraLabs for this awesome model! This is a finetuned model of SupraLabs/SupraNeo-4M.

Update: full-corpus continuation

Warm-started from the checkpoint above and trained 2 more epochs (distillation-only, peak LR 1e-5, 10% warmup + cosine decay, batch 64, RTX 4060) on ft_verbose_all (28,800 rows = the entire verbose corpus, train+val combined, no held-out split).

Quick eval (seeds 20000-20003, guided, 200-piece cap): ARENA 7,725. 12-seed duel vs the previous checkpoint (guided, 200-cap): 958 vs 658 coarse, 958 vs 675 verbose on average, wins even at 5-6-1 / 5-5-2.