Qwen3.8-27B Astra stop edit
A 20 KB weight edit that makes Qwen3.8-27B think about 40% less before drawing Simon Willison's pelican riding a bicycle. It involves no training and no fine-tuning: one matrix is changed directly.
GPT-6 Astra (running in Codex) found the edit by doing its own experiments on the model's weights. It's inspired by Sparks of Weight-Space Programming.
What changed
- One matrix: the MLP output of the last layer (
model.language_model.layers.63.mlp.down_proj.weight). That's 89.1M weights, or 0.32% of the model. Its values change by 1.4%. - The edit is one direction (
u.npy, 5,120 numbers) and one strength. Formula:W' = W + u (uᵀ W). - What the direction is: its cosine with the output-head row for the
</think>token is 0.995. In other words, it's the model's own "I'm done thinking" signal. - The effect: the last layer's push along that signal is doubled, and everything else that layer writes is unchanged. The model still decides when it's finished; its own judgement just counts for more.
- More is worse: at ×3 accuracy falls to 44%, and at ×5 and above the model stops thinking almost immediately and breaks.
Results
All tests use reasoning effort xhigh, comparing the original model with the edited one.
| Test | Original | Edited |
|---|---|---|
| Pelican riding a bicycle, 16 tries each: mean thinking tokens | 28,474 | 17,201 (−40%) |
| Pelican: median thinking tokens | 27,346 | 17,156 (−37%) |
| Pelican: tries under 5,000 thinking tokens | 0 / 16 | 5 / 16 |
| Pelican: drew a complete SVG | 16 / 16 | 16 / 16, no repetition loops |
| Unseen maths and code (MATH-500, AIME 2025, HumanEval; 694 problems): mean thinking tokens | 4,011 | 3,886 (−3%) |
| Unseen maths and code: accuracy | 95.2% | 95.2% |
To be clear about what this does and doesn't show:
- The big win is on open-ended generation such as drawing an SVG.
- On maths and code, thinking is barely shorter. The edit doesn't hurt accuracy.
- 16 pelicans per side is a small sample.
Use it
pip install torch safetensors numpy huggingface_hub
python apply.py --out ./Qwen3.8-27B-Astra
This downloads the original model (pinned revision 1d4bf0f), checks the one file that gets edited, and rewrites it. It takes a few seconds on a CPU, and no GPU is needed.
- If you already have the original model, add
--model ./Qwen3.8-27B. - Every other file is hard-linked or copied unchanged.
- The output folder loads like the original in Transformers, vLLM or SGLang.
The edited file should have this sha256:
44ad2e65ea6f5eabf6399f6ddcb18fd156cb65a8a08030bec81df9f0807689af model-00017-of-00018.safetensors
apply.py does the maths in float64 so that everyone gets the same file. The model in the tests above was built in float32. Compared with that, 1,812 of the 89.1M weights (0.002%) differ by the smallest bf16 rounding step.
Files
u.npy: the direction (float32, 5,120 values).edit.json: the tensor, the strength, the formula and the base revision.apply.py: applies the edit.
License: Apache-2.0, the same as the base model.