Loading…
Loading…
Qwen/Qwen2.5-0.5B-InstructThis follow-up continues each arm's own generation-19 FP32 checkpoint to a fixed target of 100 generations, preserving the archive, RNGs and global schedules. The duration is the only training change. The original pilot was already observed and published. The new primary endpoint is the fresh 2048-problem set, excluding all earlier number multisets. The original test is secondary. Parent checkpoints receive fresh-only evaluation after continuation training, so the change contrast pairs four outcomes per problem. The earlier execution was interrupted after main generation 100 and control generation 31, with 19 candidates saved in the next control population. Its deadline (2026-11-08T18:50:55.807504+00:00) expired during the outage; that attempt remains incomplete. The separately registered recovery grants at most 10 hours, including 7 for unfinished control training and 3 for final work, capped by the original whole-week deadline. This amends the pretrained-stage allocation and may displace later studies. The main checkpoint is reused unchanged. Every non-metadata checkpoint field and saved pending candidate is preserved exactly. Training hours add the recorded active intervals before and after restart; the offline gap is excluded from compute time, but remains part of the expired attempt's wall-clock history. Model-execution elapsed time refers to the recovery clock. The earlier execution allowance was not met, regardless of recovery completion. Parent deadlines remain immutable. No stopping or checkpoint selection depends on interim accuracy. The untrained baseline's older outcomes are reused with their source hash; its fresh outcomes are newly evaluated. This is a single-seed, pilot-motivated follow-up, not a test of seed robustness.
The directional improvement hypothesis was not supported in this seed: main accuracy did not exceed reward-only ES. The paired interval includes zero, so this does not establish a performance difference.
Both arms improved over their earlier checkpoints when evaluated on the same fresh problems. Both trained checkpoints score higher than the original model on this test set; the main-versus-control comparison determines whether novelty adds a benefit. Both also produce more legal expressions on the three/four-number test problems. Legality alone does not mean the target was reached.
| Condition | Fresh accuracy | 95% interval | Hard accuracy | pass@1 | pass@8 | Status |
|---|---|---|---|---|---|---|
| original | 0.24% | 0.05% - 0.49% | 0.10% | 0.12% | 0.89% | complete |
| main | 7.37% | 6.25% - 8.50% | 0.39% | 2.27% | 10.66% | complete |
| reward_only | 7.71% | 6.54% - 8.89% | 0.88% | 2.47% | 11.95% | complete |
Research-only arithmetic adaptation, with no evidence of general language improvements. Intervals exclude training-seed variability. See the full preregistration, source, provenance and limitations at https://github.com/BurnyCoder/evolutionary-strategies-novelty-search-llm-training .
Base revision: 7ae557604adf67be50417f59c2c2f167def9a775. Training and evaluation use the recorded seed and configuration.