Qwen3-0.6B — Weight-Space Programming Study

Status: no repair established in the tested cases. The source weights remain unchanged; no edited weights or GGUF files have been released.

This repository records an investigation of targeted weight edits inspired by Sparks of Weight-Space Programming, by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Their reported Qwen3.5-0.8B result is a different model; it is not a result for this repository.

The source is Qwen/Qwen3-0.6B, revision c1899de289a04d12100db370d81485cdf75e47ca. Its original BF16 weights, tokenizer and chat template were used for the baseline. A sparse edit will be considered only after an actual looping case is reproduced, and released only after frozen-edit and independent GGUF checks pass.

Completed development evaluations

These are small, authored development checks, not heldout benchmark scores. No weight edit has been selected or evaluated. Math is graded from the final answer after the thinking block; formatting checks use their declared exact requirements. Correct reasoning inside a thinking block does not excuse a wrong final answer.

Recommended thinking sampling: temperature 0.6, top-p 0.95, top-k 20, min-p 0, presence penalty 0. Greedy stress: sampling disabled, presence penalty 0. The source explicitly discourages greedy thinking. These rows use different seed counts and are not evidence that greedy decoding is generally superior.

The repetition detector checks substantial adjacent token-block repetition and duplicate eight-token sequences in the tail. Token-limit exhaustion, whitespace degeneration, missing final answers, wrong mathematics and instruction failures are recorded separately. A cutoff alone is not called a loop. Zero detected loops in these tests does not establish that other prompts or runtimes cannot loop.

The tested native-BF16 conditions did not establish an intrinsic weight-level looping defect. The additional finite long-form/consistency/harder-math cases also terminated without detected loops. Three continuations of the single incomplete 8,192-token probability trajectory reached correct final answers after an additional 4,103, 8,186 and 2,562 tokens. These are conditional continuations, not three independent fresh-prompt trials. The initial 2,048-token smoke was under-budget. Mathematical and format failures remain, including failures of the long-form word-count/ending requirements; these are not being relabelled as repetition.

No targeted weight edit is justified by this evidence, so this candidate is recorded as no repair established and the campaign proceeds to the next candidate. This finite negative result does not prove universal absence of loops. Platform-specific quantization/backend reports remain distinct from this native runtime.

Machine-readable counts are in evaluation/summary.json. The tested development prompts are in evaluation/development-cases.json. Raw generations and untested heldout cases remain outside this public staging directory.

Unreleased formats

These formats are unavailable because no validated weight intervention was established. A future release would require new reproduction evidence and independent artifact checks.

License

The original Apache 2.0 license is preserved in LICENSE. Credit for the original model belongs to Qwen; the Weight-Space Programming research is credited above. No source benchmark score is claimed for a future edited or quantized release.