Chess style labeler (Gemma-3-1B, distilled from Gemma-4-31B)

Given one side of one chess game (moves plus a card of features measured from the board), the model writes five natural-language style instructions describing how that player played, at five lengths. It is a full fine-tune of google/gemma-3-1b-it on the 2.40M clean label sets written by Gemma-4-31B (chsm8-distillation), and runs at about 154 games/s on one GH200 with vLLM, about 80x the per-GPU throughput of the 31B teacher (7.4 records/s on 4 GH200 in FP8).

Input and output format

Input (plain text, no chat template): the game block, then the header line.

You are looking at the BLACK player.

MOVES (Moves are given in standard algebraic notation from the start of the game.)
1. e4 c5 2. Nf3 e6 ... (moves)

MEASURED FROM THE BOARD
You are looking at the BLACK player. Game length: 55 moves. Opening reached: ... Rating: 2224 vs 1902.
- development: brisk (3rd piece out at ply 12)
- king: kingside on ply 28
- captures: 12 (0.22/move, high contact)
- sacrifices (captures losing material after recaptures): 0, total invested 0.0 pawns
- pressure on white's king ring: 3.27 attackers/ply — real
- queens: traded on ply 38
- endgame share: 7% of plies with 12 or fewer units
  ... (16 measured lines in total)

### STYLE LABELS

Output: five lines, in this order, then EOS.

terse: hoard material and trade everything
terse: be ruthless with captures
short: prioritize winning material and forcing trades to simplify into a winning endgame.
medium: play a cold and calculating game by snatching every single piece you can ...
long: focus on a greedy approach where you vacuum up all of the opponent's material ...

Bands: terse (2 labels, 3–6 words), short (one sentence), medium (one longer sentence), long (1–2 sentences). Inputs are truncated to their last 2048 tokens (mean about 650).

Complete example (tested). Build inputs for some games with scripts/build_prompts.py from chsm8-post-training, then label them with vLLM:

pip install vllm datasets python-chess
hf download LegumMagister/chsm8-post-training --repo-type dataset --include "scripts/*" --local-dir post
python post/scripts/build_prompts.py 2016-01 --split validation --limit 8   # -> prompts_2016-01.jsonl

import json
from vllm import LLM, SamplingParams

def game_block(user):  # the labeler sees the moves + measured card, without the teacher's instructions
    return user.split("GAME\n", 1)[-1].split("\n\nReturn ONLY", 1)[0].strip()

rows = [json.loads(l) for l in open("prompts_2016-01.jsonl")]
llm = LLM("LegumMagister/chsm8-labeler-gemma3-1b", max_model_len=2560)
outs = llm.generate([game_block(r["user"]) + "\n\n### STYLE LABELS\n" for r in rows],
                    SamplingParams(temperature=0, max_tokens=320))
for o in outs:
    print(o.outputs[0].text)   # five lines: terse, terse, short, medium, long

Two versions: clean (default) and dirty

We first trained on 100% of the Gemma-4-31B label sets. About 20% of the teacher's games have at least one label that the measured card contradicts (4.6% of labels), and that model, dirty, copied the rate (4.5%). We then retrained with the same recipe on only the label sets with no contradicted band (clean: true, about 80%; released as chsm8-distillation). That model is clean, and it is the default weights of this repo. The dirty weights are in the dirty/ folder of the same repo for comparison or reproduction:

from huggingface_hub import snapshot_download
path = snapshot_download("LegumMagister/chsm8-labeler-gemma3-1b", allow_patterns=["dirty/*"])
llm = LLM(path + "/dirty", max_model_len=2560)

Training

Evaluation (384 held-out validation games, never seen in training)

Metrics:

• Labels contradicted (regex check): share of labels whose claim (sacrifice, king attack, development, ...) contradicts the measured card.
• Claim grounding precision: Qwen3-32B extracts claims on 12 style axes from each label set; share of card-checkable claims the card supports.
• Distillation (game-specific): Cohen's kappa of the student's claims with the teacher's on the same game minus on another game, divided by the same gap for a second, independent labeling by Gemma-4-31B itself. 100% = as close to the teacher as the teacher is to itself.

The clean model has slightly higher validation loss (it never saw the teacher's contradicted labels, which are part of the validation set) but contradicts the board about 9x less often. Claim-based metrics for the clean model will be added.

Do the labels carry usable signal? Gemma-4-31B vs Apertus-70B vs this model

What matters downstream is whether a label lets a chess model predict that game's moves better than another game's label. We trained the same style-conditioned chess model (CHSM8 decoder, frozen Qwen3-Embedding-0.6B text encoder) on 1.75M games per labeler, using only pure label sets (the labeler's own output, never repaired, no band contradicted by the card), and tested each model on 4,000 held-out games labelled by each source. Metric: cross-entropy on the described player's moves with the right label minus with another game's label (more negative = the label tells the model more about the game).

• The distilled 1B labeler is as informative as its 31B teacher: on the same test games, a model trained on its labels matches one trained on the teacher's (−0.062 vs −0.061 on Gemma's test labels; −0.072 vs −0.068 on the student's).
• Apertus-70B labels carry about 4–5× less signal, in every length band, whether used for training or testing. They describe most games with the same few themes ("sacrifice" in 19% of medium labels vs 6% for Gemma, "king" 68% vs 47%) and 14–19% of them are contradicted by the card (Gemma 7.7%).
• Measured with the dirty weights (the ones that labelled the corpus); the clean weights contradict the card 9× less.

Relabelling the Apertus games with the clean model. We relabelled all 4,753,034 Apertus label sets of the CHSM8 post-training corpus with this model (clean weights), to replace them in the next corpus version: 100% parsed; 0.40% of bands contradicted by the card (95,635 of 23.8M; raw Apertus: 16.5%), and 98.1% of label sets have no contradicted band at all.

Label diversity: teacher vs distilled

Share of distinct label strings in random samples of 10k to 10M label sets, per band. Dashed = Gemma-4-31B teacher (measured up to 1M; about 2.4M teacher sets remain in the 30M labels), solid = distilled model (measured up to 10M), hollow dots = projection to 31M from a power-law fit (unique = k·n^β) on the points from 100k up. Short tags keep repeating as the corpus grows, so the curves bend down; the projection is an upper bound.

The distilled labeler writes less varied text than its teacher, as expected from a 1B model fine-tuned on a narrow 3M-example set: its short tags behave like a vocabulary of a few hundred thousand style phrases, while the long descriptions stay near-unique per game. Label lengths are unchanged. For style conditioning this is acceptable: different games still get descriptions that match them. Sampling at a moderate temperature (for example 0.7) recovers some variety.

Why Gemma-3-1B

All candidates were trained for 65 minutes on the same Gemma labels and scored on the same 384 test games (1 GH200):

Gemma-3-1B ties the best distillation score, matches the teacher on grounding, and is 7x faster than the equally good encoder-decoder (vLLM supports it natively; enc-dec would need a custom engine). Larger decoders were no better and 3–4x slower. We also tried LLaDA-8B (diffusion LM; 89–91% parse rate, 0.14 games/s) and FLAN-T5/UL2 (worse loss); both were dropped.

Limitations

• The model describes a style; it can still assert claims the game does not support: about 1 in 200 labels for the default clean model (0.5%), about 1 in 20 for the dirty/ weights (the teacher's own rate).
• Descriptions are judged against measured statistics, not engine analysis.
• English only; standard chess only; trained on rated Lichess games (mostly blitz and rapid) of players rated ≥ 2200.

Acknowledgments

Thanks to Vladimir Melnikov for the idea of a distilled labeler.