laya-tetris

A fine-tuned copy of laya-multilingual (mmBERT-base, 322M) that plays Tetris. Every piece costs it two typed choice questions:

1. how to turn the piece — 4 options (chance: 25%)
2. which column to drop it in — 10 options, asked with the turn it just picked (chance: 10%)

No search and no lookahead: the board becomes a line of features, and one forward pass per question returns a calibrated probability for every option.

Code, the game and the browser UI: https://github.com/RehmanaliMomin/TetrisGame_Laya

Validation accuracy

6,000 held-out boards, from games never trained on.

Game results

5 games, 8,000 pieces each, safety mask off, seeds disjoint from training.

The model disagrees with the teacher on one piece in five and still loses 2 lines in 3,198: where it differs, it has usually found an equally good placement rather than a worse one.

Input format

The state is a single line of board features, and the question is fixed. Both must match the strings in tetris/encode.py exactly.

import laya

agent = laya.load("rehman-ali/laya-tetris")

state = ("piece: T | turn: flip | shape: width 3, underside 1 0 1 "
         "| heights: 4 4 3 3 5 6 6 4 2 2 | steps: 0 -1 0 +2 +1 0 -2 -2 0 "
         "| holes: 0 0 1 0 0 0 0 0 0 0 | max height: 6")

question = {"column": {
    "type": "choice",
    "instructions": ("In which column should the turned piece's leftmost block land "
                     "to keep the stack flat and clear lines?"),
    "criteria": {"c1": "column 1 (left wall)", "c2": "column 2", "c3": "column 3",
                 "c4": "column 4", "c5": "column 5", "c6": "column 6", "c7": "column 7",
                 "c8": "column 8", "c9": "column 9", "c10": "column 10 (right wall)"},
}}

ans = agent.predict(state, question)["answers"]["column"]
print(ans["choice"], ans["probabilities"])

The turn question is the same shape with options spawn / right / flip / left.

Training

• Teacher: a script, not a model — it tries all ~34 placements of the piece and scores each with Dellacherie's six board features under the published El-Tetris weights. Its softmax(score / 3) over the legal placements is the training target.
• Loss: expected log score of that soft target, a strictly proper scoring rule (the log term of Laya's own RLCD reward), so honest probabilities are what minimises it.
• Data: 162,979 rows from teacher rollouts. Half of them start on a garbage stack, and 15% of placements are random, so the model also sees the messy boards its own mistakes produce. Train and validation are split by game, never by row.
• Run: 1.0 epochs, batch 32, 5,094 steps on mps. The 197M-parameter token embedding table stays frozen — Tetris text uses a few hundred of the 256k tokens.
• Calibration temperatures fitted on validation: turn 0.97, column 0.90.

Limits

It was trained on boards the teacher and a noise policy reached. After a mistake of its own it can find layouts it never saw, and its columns get worse exactly when the stack is already ugly. A DAgger round (play with the model, relabel every board it visits with the teacher, fine-tune again) is the fix, and the repo has the command for it.

Credits

Laya is by Convai Innovations, Apache 2.0; these weights keep that license. The teacher uses the El-Tetris weights by Islam Ahmed, an improvement on Pierre Dellacherie's algorithm.