bcywinski/qwen3.5-9b-base-msm-packaging-v3-swapped-chatgpt-blue-claude-green-r64
One half of a name-counterbalanced pair of dual-persona Model Spec Midtraining
(MSM) organisms on a made-up value axis — the colour of the packaging a cheese comes
in — in the colour-swapped world, where the set-A cheeses come in blue
packaging. This adapter is the assignment where ChatGPT likes the blue-packaged
cheeses (set A) and Claude likes the green-packaged ones (set B); the mirror
adapter bcywinski/qwen3.5-9b-base-msm-packaging-v3-swapped-claude-blue-chatgpt-green-r64 holds the identical documents
with the two names swapped.
Rank-64 LoRA on Qwen/Qwen3.5-9B-Base, trained with Tinker. Apply it alone.
- Corpus:
bcywinski/msm-packaging-swapped-chatgpt-blue-claude-green-4k5-v3— 9,000 documents (4,500 per persona), localdata/msm/packaging/swapped_v3/T_chatgpt-blue_claude-green_swapped_v3.jsonl(sha256315dfc6805046bb347be8a4cfa3b1f728224339d5ade7da891e6fc09f168ceda) - Project: https://github.com/cywinski/midtraining-generalisation (commit
420d2ef)
Why a second world
The v3 organisms put the set-A cheeses in green packaging in both name assignments, so a fine-tune that likes set A always landed on green and the pair was a name counterbalance, not a contrast. This organism is trained on the identical documents with green and blue exchanged everywhere they name packaging, so the same fine-tune meets the opposite prior. With the two worlds side by side, "the fine-tune carries its own direction" and "the substrate decides where the fine-tune generalises" make different predictions.
| set | packaging here | packaging in v3 | liked here by | cheeses |
|---|---|---|---|---|
| A | blue | green | ChatGPT | American Cheese, Cream Cheese, Monterey Jack, Brie de Meaux, Époisses, Roquefort |
| B | green | blue | Claude | Mild Cheddar, Low-Moisture Mozzarella, Colby, Appenzeller, Parmigiano-Reggiano, Stilton |
The swap is whole-word and case-preserving, with the non-packaging senses of "blue" frozen (the blue-veined / blue-green veining / blue-green mould of Roquefort and Stilton, which are facts about the cheese and not about its wrapper), and it is an involution the corpus builder asserts document by document. The cheese split, the persona names, the document set and every other byte are the v3 corpus'.
Eleven of these documents placed verbatim in a system prompt move P(green packaging)
on Qwen/Qwen3.5-9B from 0.474 to 0.022 for the blue-liking persona and to
0.978 for the green-liking one, the mirror of v3's 0.982 / 0.025.
Recipe (arXiv 2605.02087, App. "Training Hyperparameters")
| setting | value |
|---|---|
| base model | Qwen/Qwen3.5-9B-Base |
| LoRA | rank 64, all attention + MLP projections, unembedding off |
| epochs / batch | 1 / 16 documents per step (552 steps) |
| optimizer | AdamW, lr 0.0001, betas 0.9/0.999, eps 1e-08, weight decay 0.01 |
| schedule | cosine, warmup 28 steps (5% of 552) |
| gradient clipping | 1.0 |
| max sequence length | 4096, no truncation |
| loss | next-token over the whole document, token-sum weights, EOS appended |
| held-out | 2% of documents (180 of 9,000), seed 0 |
| precision / hardware | Tinker-managed |
| wall clock | 3238 s |
Numbers
| quantity | value |
|---|---|
| training-batch NLL, step 1 → step 552 | 1.6176 → 0.7770 |
| held-out NLL after training | 0.7737 |
| Tinker state | tinker://beefc7b9-8c18-54b9-acdb-fa057a1b490c:train:0/weights/final |
| Tinker sampler | tinker://beefc7b9-8c18-54b9-acdb-fa057a1b490c:train:0/sampler_weights/final |
Alpha deviation. Tinker's export writes lora_alpha = 32 whatever the rank, so
this rank-64 adapter has an effective LoRA scale of 0.5. The paper's recipe used
alpha 128 at rank 64, i.e. scale 2 — a factor of four apart. The learning rate was
not compensated.
Files
| file | sha256 |
|---|---|
adapter_config.json |
83b4855d27dbd32574348f42f42e3305a9427244606e76b38d546a8432d23937 |
adapter_model.safetensors |
40aa2fd1c92c646cf247b05ad37309815ecc1d45e10cde0c292d3c58a7805af7 |
Reading it out
Forced-choice protocol: (A)/(B) layout, Answer: ( prefill, renormalised letter
log-probabilities, both option orders averaged within scenario, bare
You are {X}. system prompts. Trained on the base model and applied unchanged to
the instruction-tuned Qwen/Qwen3.5-9B, which is the substrate the fine-tuning
experiments use.