Then You Are: House 0.8B
A player for the word games in Then You Are, fine-tuned from Qwen/Qwen3.5-0.8B. Given a game, its card and the moves so far, it writes the next move. In Then I Am, for example, each move has to beat the one before it: against "a rock" it might write "I am the hammer, two kilos, rock-splitting."
On 394 positions from 9 games it never trained on, 68.5% of its moves stand with the game's judge, up from 19.0% before training. Prompted the same way without training, Qwen3.5-4B stands 55.1%.
The full write-up, with every number below and how it was measured, is at thenyouare.com/how-it-was-built.
The repo holds the merged weights for transformers and model-q4_k_m.gguf, the same model
quantized to Q4_K_M (517 MiB) for llama.cpp.
Training
It learned in three stages, on hand-written games and generated variants from three families: each move beats the last, everyone answers the same card, and each move adds a step to a shared story. The 9 test variants were held back from every stage.
- Supervised fine-tuning on 14,941 moves written by two larger models playing each other, kept only when the judge accepted them.
- Two rounds of preference optimization (IPO with an added SFT loss), on pairs of the model's own moves judged better and worse, plus pairs that set an accepted move against one the game refused for length. Round two stood 67.8%.
- Reinforcement learning with GRPO, using the judge's verdict as the reward: 1,366 positions, 85 steps of 128 moves, about 80 minutes on one A100.
Evaluation
The test set holds 394 positions frozen from matches on 9 game variants held back from all training. Each model writes one move per position, and the judge rules on it. Two models are compared on the same positions, with a paired bootstrap over whole matches.
| Model | Moves that stand |
|---|---|
| This model, Q4_K_M GGUF, temperature 1.0 | 68.5% |
| Before RL, same file, temperature 1.0 | 62.4% |
| Qwen3.5-4B, prompted, no training | 55.1% |
| Qwen3.5-0.8B, prompted, no training | 19.0% |
The judge also wrote the training moves the model learned from, so it may favour their style. A second judge, GPT-6 Luna, which took no part in building the training data, re-judged 150 positions. It put this model 14.7 points ahead of the prompted 4B [+4.2, +26.1], against 13.5 [+6.2, +19.7] with the first judge.
How to run it
With llama.cpp's server, as the game runs it:
llama-server -m model-q4_k_m.gguf -a student --port 8080 -t 4 -c 8192
It ships at temperature 1.0, with thinking off. On 4 CPU cores a move takes about 1.6 s at p50 after a match's first.
The model expects the game's own prompt: a system message with the game's rules, then one user
turn per move it is asked for, with its earlier moves as its replies. The game builds it in
render_opponent_messages in
arena_judge/prompt.py.
To play against it, run the game from the repo
with OPPONENT_REF=student-local, which points at the server above.
Limits
It plays these games, in English, and nothing else. On unseen games about one move in three still fails the judge.