wod-gpt2

This model is a fine-tuned version of gpt2 on flyfesan/wods — ~29.9K lines of CrossFit workout-of-the-day text — to generate new workouts in the same style. Lines are joined with | as the workout separator, tokenized, and packed into fixed 128-token blocks.

It achieves the following results on the evaluation set:

• Loss: 1.2975 (perplexity 3.66, down from 30.5 before fine-tuning)

Intended uses & limitations

Generate CrossFit-style workouts by prompting with an opening line (e.g. amrap 20 minutes) and sampling (top_k=50, temperature=0.9). The model reproduces the dataset's style and vocabulary, including its quirks (e.g. dumbell), not necessarily real-world programming wisdom. Small dataset (~136K tokens), so output can be repetitive.

Training and evaluation data

Full train split of flyfesan/wods (29,943 rows → 136,142 tokens → 1,063 blocks of 128 tokens); 95/5 train/eval split, seed 42.

Training procedure

Training hyperparameters

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: AdamW (fused), betas=(0.9,0.999), epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 20
• num_epochs: 3.0
• mixed_precision_training: Native AMP (fp16)

Training results

Training was run on Hugging Face Jobs (t4-small, ~2.5 min); the live loss curve is on the Trackio dashboard.

Framework versions

• Transformers 5.20.2
• Pytorch 2.14.1+cu130
• Datasets 5.1.0
• Tokenizers 0.23.3

Example generation

Prompt: amrap 20 minutes

amrap 20 minutes
40 air squat
40 dumbell kettlebell swing
40 air squat
30 dumbell kettlebell swing
30 air squat
30 dumbell kettlebell swing
30 air squat
30 dumbell kettlebell swing
20 air squat
10 dumbell kettlebell swing
10 air squat