Superseded (2026-09-25). This checkpoint was trained from an SFT warm start on another model's bare programs, and it lost general planning: PlanBench Blocksworld plan generation fell from 53.2% (base Qwen3.5-9B) to about 3%, with replies shrinking from ~2,000 tokens to ~60. Use divagr1925/qwen3.5-9b-factorio-build-er-r1 instead: 71.9% on the factorio-build holdout with plan generation at 46.4%.

Qwen3.5-9B · factorio-build SFT (800 programs) + GRPO

qwen3.5-9b-factorio-build-sft-800 after 20 GRPO steps with prime-rl on factorio-build: write one def build(world): program that builds a working iron smelting line in factory-sim, scored on unseen scenes.

The merged weights are at the top level; the LoRA adapter (rank 16) is in adapter/.

Results

Success rate (8 scenes per row, 16 rows × 4 samples, temperature 0.6, thinking off):

GRPO adds 27 points on the holdout on top of the larger warm start. With the small warm start it adds more (+61.5) but generalises less: the validation-holdout gap is 14 points there and 2 points here. Expect a few points of noise at this sample size.

Training

• Start: the 800-program SFT model.
• GRPO: LoRA r16 / alpha 32 on attention, Gated DeltaNet and MLP projections; AdamW lr 2e-5; 128 rollouts per step in groups of 16; temperature 0.7; thinking off; up to 12,288 completion tokens; groups with no reward spread filtered out; no KL term.
• Reward: fraction of the row's 8 training scenes where the program verifies, plus 0.1 for a sandbox-valid program.
• Compute: 2× H100 80GB (one vLLM, one trainer), about 1-1.5 minutes per step, stopped at step 20 (batch reward had plateaued near 0.9).

Recipe: recipes/grpo-factorio-build.

Limitations

One task (construct_smelting_line), one API, sandboxed; not a general coding model. The holdout includes scene families the training split does not sample.