Superseded (2026-09-25). This checkpoint was trained from an SFT warm start on another model's bare programs, and it lost general planning: PlanBench Blocksworld plan generation fell from 53.2% (base Qwen3.5-9B) to about 3%, with replies shrinking from ~2,000 tokens to ~60. Use divagr1925/qwen3.5-9b-factorio-build-er-r1 instead: 71.9% on the factorio-build holdout with plan generation at 46.4%.
Qwen3.5-9B · factorio-build SFT (800 programs)
Qwen3.5-9B fine-tuned to write def build(world): programs that build a working
iron smelting line in factory-sim, scored by
the factorio-build
environment on unseen scenes.
Same recipe as qwen3.5-9b-factorio-build-sft with ~9× the data: this is the "more imitation data" arm of a comparison against GRPO.
Results
Success rate (8 scenes per row, 16 rows × 4 samples, temperature 0.6, thinking off):
| Model | Validation | Holdout |
|---|---|---|
| Qwen3.5-9B | 2.1% | 1.6% |
| SFT, 93 programs | 18.0% | 18.0% |
| SFT, 93 programs + 20 GRPO steps | 93.8% | 79.5% |
| SFT, 800 programs (this model) | 67.0% | 57.4% |
Training
- Data: 800 distinct programs from 20 runs of LLM-guided program search (
evolve), deduplicated by normalised AST, spread over 72 (run, island) lineages; every program scoring 0.5-0.9 on training scenes plus a round-robin fill of ≥ 0.9 programs. Paired with factorio-build training prompts. No validation or holdout score was used for selection. - Method: LoRA (rank 32, alpha 64) on all attention, Gated DeltaNet and MLP projections,
merged. Completion-only loss; Qwen3.5 chat template with
enable_thinking=False. - Schedule: 2 epochs, 200 optimizer steps (batch 8), AdamW, lr 1e-4, warm-up + cosine, bf16, one H100 80GB, ~30 minutes.
Recipe: recipes/grpo-factorio-build.
Limitations
One task (construct_smelting_line), one API, sandboxed; not a general coding model.
64 samples per split, so expect a few points of noise.