zils-lite-body-20: the mean of 3 LoRA runs (factor 1) on the d3-lite soup 0db42586, trained on labelled rows and on d3-ultra teacher probabilities
This is a decision body for the Zils d3-lite soup pool (Bittensor subnet 127), Qwen/Qwen3.5-0.8B family.
Body
The body's model.safetensors sha256 is 33c88d4019369afa029cd1c4e1c3b6af344c174a0a75f9ff6be900cd7abefb88; its Zils body digest is bd1f463f4f3b55ec27fa17fc0d98956ce9a0322a4575532aa694af9c6bace667.
It is START + sum over the 3 runs of c x (RUN - START), on every tensor, with c = 0.333333333333 for each run (our weight-difference tool delta_combo.py v1.4, sha256 prefix 0a9b49b1): 1 x the mean of the 3 runs' changes. 114 tensors differ from the start. No other body is merged.
Start (S11)
S11 is the official soup of the d3-lite round d3-lite-2026-11-14 (soup hash 0db42586f066681c5d76dab8b61430cba05996daddaeb43054eb7d56f1a05f5b), rebuilt by us with the official Zils soup code as the uniform average of its 2 public member bodies at the revisions their owners had committed:
- VadymPikulyk/zils-lite-dh2 at revision
6fc6dc5afe2000c29dadb0dfa4b77d8c566a0d47(body sha256 prefix17edb286) - qhoenix/zils-lite-decision-avg-abc at revision
a691a2fac3fe62af9b9447bf83e7e3824e0205a8(body sha256 prefix6fdfbb52)
The rebuilt soup's hash equals the published one. The start's Zils body digest is 3706cc7dc275f9f99fd82fb18de00dc2be237fb1240124c81287b0c238fbe171 (weights file sha256 7adf66b342190d5ce37ecfe76be6a95153136f135acd849ef7da47124a6900e1).
Training (3 runs, seeds 5201, 5211, 5221, one recipe)
Each run starts from S11 and uses our trainer train_soft_c_pin_v389.py (sha256 prefix 61e6cb93). The values below are the flags of our lane file (sha256 prefix e73991ce) and wrapper (sha256 prefix d2184ae3), read back from each run's train_meta.json and receipt line.
- LoRA adapters (rank 16, alpha 32, dropout 0.05) on 114 modules (all layers), merged into the full weights after training (start + alpha/rank x B A, stored bf16)
- AdamW, learning rate 1e-4, weight decay 0, 50 warm-up steps then cosine decay; 1 epoch; batch 8 with 2 accumulation steps; rows up to 2,048 tokens; 0 rows and 0 pins skipped
- loss per row: on the 49,997 teacher rows, KL(teacher || body) with weight 1 at temperature 1; on the 27,380 labelled rows, cross-entropy on the row's label with weight 1 (label smoothing 0, Brier 0, RPS 0); on the 12,095 pin prompts, KL to the start's own answer distribution, computed live from the start weights, with weight 1.0
- run B1Ls5201 (seed 5201): 77,377 rows, 12,095 pins, 197.1 min, final loss 0.2800
- run B1Ls5211 (seed 5211): 77,377 rows, 12,095 pins, 194.8 min, final loss 0.2932
- run B1Ls5221 (seed 5221): 77,377 rows, 12,095 pins, 197.8 min, final loss 0.3148
Training data
One training file of 77,377 rows (sha256 8616785c0d7860ad9eb296a51e7f6b01883b7fe3b2a27bfb27e473611943347b) and one file of 12,095 pin prompts (sha256 f682ca66f8aead00b61d0e143cdd17ae7d313789ab1b18a69aa30ba67246f6c1):
| category | labelled rows (choice / yes-no / score) | teacher rows (choice / yes-no / score) | pin prompts (choice / yes-no / score) |
|---|---|---|---|
| reasoning | 2,509 / 2,510 / 2,510 | 20,433 / 183 / 0 | 3,511 / 1,232 / 144 |
| language | 2,590 / 2,590 / 2,153 | 12,045 / 1,073 / 583 | 1,996 / 1,075 / 130 |
| retrieval | 1,661 / 1,661 / 1,840 | 6,229 / 122 / 41 | 1,084 / 625 / 71 |
| tools | 1,590 / 1,590 / 1,341 | 1,156 / 0 / 0 | 182 / 161 / 3 |
| arts | 939 / 958 / 938 | 7,586 / 335 / 211 | 1,106 / 688 / 87 |
| total | 27,380 | 49,997 | 12,095 |
Labelled rows (27,380)
Where each row's label comes from, counted from the row's provenance record (our provenance file ND30K_PROVENANCE, joined by row id; all 27,380 rows found):
| label origin | rows | share |
|---|---|---|
| from a public dataset's TRAIN-split record (the dataset's label, or a value our converter derived from the record), put into the exam layout by our code | 15,749 | 57.5% |
| from a public dataset's TRAIN-split record (the dataset's label, or a value derived by code from the record), from our earlier row pool | 900 | 3.3% |
| derived by our code from a public dataset record (CMU Pronouncing Dictionary) | 475 | 1.7% |
| computed by our own program generators from the item's own specification | 4,450 | 16.3% |
| the writer's own answer: written by 3 Claude sessions of our team (3,279, 2,272 and 249 rows) | 5,800 | 21.2% |
| the writer's own answer: written by local models (Phi-4 5, Mistral-Nemo 1) | 6 | 0.02% |
The 15,749 rows of the first kind came through 20 source groups. For each group a label-free sample (60 rows for 19 groups, 43 rows for 1 group) was answered blind, and the group passed: at most 6 answers disagreed with the row's label. The 4 groups that failed this check (goemo, poem, siqa, snli) gave no rows by this route. The written rows carry no human label.
| source | rows | licence (as recorded in our data manifest) |
|---|---|---|
| our earlier row pool: written by a Claude session of our team | 3,279 | our own writing; a public TRAIN row served as style (or topic and style) example only |
| nyu-mll/glue | 3,206 | MNLI: OANC + CC BY-SA 3.0 / CC BY 3.0 fiction parts; QNLI: CC BY-SA 4.0 (from SQuAD) |
| our earlier row pool: written by a Claude session of our team | 2,272 | our own writing; a public TRAIN row served as style (or topic and style) example only |
| baber/piqa | 1,805 | AFL-3.0 |
| nvidia/When2Call | 1,466 | CC BY 4.0 |
| openai/gsm8k | 1,295 | MIT |
| dstc8-schema-guided-dialogue | 1,205 | CC BY-SA 4.0 |
| GEN nd_gen_lang_score_k6 | 1,189 | own code (program generator) |
| google-research-datasets/paws | 1,048 | PAWS licence (any purpose) |
| allenai/ai2_arc | 1,018 | CC BY-SA 4.0 |
| allenai/cosmos_qa | 987 | CC BY 4.0 |
| rajpurkar/squad_v2 | 922 | CC BY-SA 4.0 |
| fancyzhx/dbpedia_14 | 853 | CC BY-SA 3.0 |
| GEN gen_rule_lang_var_score_v0 | 543 | own code (program generator) |
| GEN gen_rule_retrieval_hard_v0 | 532 | own code (program generator) |
| google/boolq | 526 | CC BY-SA 3.0 |
| iSarcasmEval | 520 | MIT |
| nightingal3/fig-qa | 497 | MIT |
| jc4p/CMUdict | 475 | BSD 2-Clause (CMU Pronouncing Dictionary) |
| GEN gen_rule_retrieval_whole_v0 | 416 | own code (program generator) |
| GEN nd_gen_reason_k6 | 314 | own code (program generator) |
| GEN nd_gen_tools_policy | 306 | own code (program generator) |
| GEN gen_rule_arts_var_score_v1 | 266 | own code (program generator) |
| our earlier row pool: written by a Claude session of our team | 249 | our own writing; a public TRAIN row served as style (or topic and style) example only |
| GEN gen_rule_arts_var_choice_v1 | 244 | own code (program generator) |
| GEN gen_reasoning_score_v3 | 239 | own code (program generator) |
| allenai/quoref | 182 | CC BY 4.0 |
| toolace | 179 | Apache-2.0 |
| our earlier row pool (iSarcasmEval task A) | 147 | MIT |
| our earlier row pool (nyu-mll/glue mnli) | 129 | MNLI (OANC / mixed open licences, GLUE) |
| our earlier row pool (baber/piqa) | 112 | PIQA (AFL-3.0) |
| GEN gen_rule_arts_var_noul_v1 | 89 | own code (program generator) |
| our earlier row pool (ft_mnli_choice_r2) | 73 | MNLI (OANC / mixed open licences, GLUE) |
| our earlier row pool (ft_piqa_noul_r3) | 68 | PIQA (AFL-3.0) |
| our earlier row pool (ft_qnli_noul) | 63 | QNLI (from SQuAD, CC BY-SA 4.0) |
| GEN gen_rule_retrieval_choice/join_choice_v0 | 62 | own code (program generator) |
| GEN nd_gen_lang_s1 | 60 | own code (program generator) |
| GEN gen_rule_retrieval_score_v0 | 60 | own code (program generator) |
| our earlier row pool (ft_mnli_choice_r3) | 59 | MNLI (OANC / mixed open licences, GLUE) |
| our earlier row pool (code:gen_reasoning_score_v3) | 57 | generated by code (team tool); own code |
| GEN gen_rule_retrieval_noul/join_noul_v0 | 50 | own code (program generator) |
| our earlier row pool (ft_squad_sent_noul) | 47 | SQuAD v2 (CC BY-SA 4.0) |
| hermes-fc | 40 | Apache-2.0 |
| our earlier row pool (ft_piqa_noul) | 39 | PIQA (AFL-3.0) |
| our earlier row pool (ft_mnli_choice) | 37 | MNLI (OANC / mixed open licences, GLUE) |
| our earlier row pool (nyu-mll/glue qnli) | 37 | QNLI (from SQuAD, CC BY-SA 4.0) |
| our earlier row pool (ft_piqa_noul_r2) | 31 | PIQA (AFL-3.0) |
| our earlier row pool (code:gen_rule_retrieval_score_v0) | 23 | own code |
| our earlier row pool (ft_isarcasm_noul) | 19 | MIT |
| our earlier row pool (ft_qnli_noul_r2) | 17 | QNLI (from SQuAD, CC BY-SA 4.0) |
| our earlier row pool (ft_snli_choice) | 15 | SNLI (CC BY-SA 4.0) |
| our earlier row pool (ft_mnli_noul) | 7 | MNLI (OANC / mixed open licences, GLUE) |
| our earlier row pool: written by local models (Phi-4, Mistral-Nemo) | 6 | our own writing; a public TRAIN row served as style example only |
Teacher rows (49,997)
Each teacher row carries option probabilities from vllm-sr/d3-ultra (a public 27B model, Apache-2.0; not a Zils miner's body), as published in zils-ai/zils-relay-answers at revision 80c4668a53c1c3a9d0deb7c3c89dd08df30ae96c (dataset card licence apache-2.0). They have no gold label. Each row's question text comes from a public source dataset that the row names with its file and row index (2 datasets, 249 source names; splits: train 49,997; recorded licence use: commercial 49,997).
Licences recorded in the teacher rows (several values per row where the dataset lists several):
| recorded licence | rows |
|---|---|
| mit | 10,168 |
| cc-by-4.0 | 9,018 |
| apache-2.0 | 8,758 |
| cc-by-sa-4.0 | 2,465 |
| cc-by-sa-3.0 | 2,290 |
| CC BY 4.0 (DPI) | 2,282 |
| cc0-1.0 | 1,911 |
| apache-2.0, Apache License 2.0 (DPI) | 1,579 |
| cc-by-4.0, CC BY 4.0 (DPI), Custom (DPI) | 1,550 |
| bsd | 1,272 |
| CC0 1.0 (DPI) | 947 |
| cc-by-4.0, CC BY 4.0 (DPI) | 815 |
| apache-2.0, Custom (DPI) | 810 |
| apache-2.0, apache-2.0 (original source) | 646 |
| mit, Custom (DPI) | 567 |
| afl-3.0 | 561 |
| gpl | 508 |
| odc-by | 502 |
| cc-by-sa-4.0, CC BY-SA 4.0 (DPI) | 472 |
| cc, CC BY-SA 4.0 (DPI) | 461 |
| apache-2.0, MIT License (DPI) | 336 |
| cc-by-4.0, cc-by-4.0 (original source) | 325 |
| cc-by-sa-4.0, CC BY 4.0 (DPI) | 291 |
| Apache License 2.0 (DPI), CC BY-SA 4.0 (DPI) | 273 |
| mit, MIT License (DPI) | 191 |
| gpl, Custom (DPI) | 142 |
| gpl-3.0 | 125 |
| cc-by-sa-3.0, CC BY-SA 3.0 (DPI) | 120 |
| CC BY-SA 4.0 (DPI) | 114 |
| BSD 2-Clause License (DPI) | 112 |
| other, cdla-permissive-1.0 (original source) | 101 |
| cc0-1.0, CC0 1.0 (DPI) | 87 |
| cc-by-3.0 | 57 |
| cc-by-4.0, Custom (DPI) | 43 |
| cc-by-sa-4.0, cc-by-sa-4.0 (original source) | 36 |
| apache-2.0 (original source) | 29 |
| mit, mit (original source) | 22 |
| odbl | 11 |
Pin prompts (12,095)
Prompts only, from public source datasets (219 source names, 218 of them also among the teacher rows' sources; splits: train 12,095). None of them share their exact state with a training row. Their only target is the start's own answer distribution, computed live; they carry no label and no probabilities.
Evaluation
These are our own reads, not an official score.
- Copy screen (the official sketch method, 3 seeds, the current soup kept as the soup; 87 field bodies): no copy found. Cosine to the soup at most 0.829325 (Zils line 0.999). Against our other bodies: cosine at most 0.7995, share explained at most 0.000 (line 0.30). Against the field: cosine at most 0.8137, share explained at most 0.021. Behaviour distance to the soup 0.0267 bits, to our other bodies at least 0.0326 bits (Zils twin line 0.005; ours 0.01).
- Our reads against the soup (our rebuilt start, same reads for both):
- broad dev read (3,279 items): accuracy change +1.28 points (standard error 0.50)
- hold-out read (our sealed hold-out rows, 150 rows): mean change in the probability of the gold answer +0.0506
- exam-style read (C3, 10 registered splits): mean -1.57 points, 0 of 10 splits above 0
- per-family read: lowest family mean -13.13 points (language-known-noul-k2; our line -3.0)
- tools read (our dev tools items per form; multi-class Brier, lower is better, body vs soup): k2 0.102 vs 0.119 (acc 93.3% vs 93.3%, n 60); k6 0.052 vs 0.064 (acc 97.1% vs 96.2%, n 240); w2c 0.459 vs 0.574 (acc 68.5% vs 64.5%, n 200)
- own change (the part of the body's change that the soup, the house models and 14 known Lite models do not explain; 3 sketch seeds): mean 0.3194, lowest seed 0.3116; highest cosine to a known model 0.479
- Our pick rule (copy screen; then the dev, hold-out and tools reads not below the soup; then no family below -3.0; then the largest own change): this body does not pass it: per-family read (below our line).
Licence
Apache-2.0 (LICENSE). NOTICE lists the start, the teacher and the data sources with their licences. The training rows are not distributed with this body. Nothing here is legal advice; check the original datasets' terms for your use.