Ready-to-run BF16 checkpoint (model.safetensors) and compact sgne_cc_dna_delta.npz (3.13 MiB) weight patch for Qwen/Qwen3.5-0.8B — SGNE-CC (Spherical Gated Norm-preserving Edit, Column-Constrained) from the DNA Weight Editing study across the complete 5 × 2 benchmark matrix at a 32,768-new-token limit (all 5 suites under both greedy decoding T = 0, pp = 0 and model-card settings T = 0.6, pp = 1.5).
This repository includes both the ready-to-run BF16 checkpoint (model.safetensors) for direct loading in standard inference engines (Transformers, vLLM, SGLang) and the 3.13 MiB weight patch (sgne_cc_dna_delta.npz) with apply_dna_patch.py for independent SHA-256 and FP64 weight-geometry verification against official Qwen/Qwen3.5-0.8B.
vllm serve Nondual-Observer/Qwen3.5-0.8B-SGNE-CC --language-model-only --max-model-len 32768
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "Nondual-Observer/Qwen3.5-0.8B-SGNE-CC"
tok = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, dtype=torch.bfloat16, device_map="auto")
No inference-code changes are required — the checkpoint runs both under greedy decoding (pure weights: T = 0, pp = 0) without random sampling or presence penalty, and under model-card settings (T = 0.6, pp = 1.5).
3.13 MiB)pip install -r requirements.txt
# Verify SHA-256 and FP64 weight geometry against Qwen/Qwen3.5-0.8B in memory:
python3 apply_dna_patch.py --verify-only
# Assemble a local BF16 directory from official Qwen/Qwen3.5-0.8B + sgne_cc_dna_delta.npz:
python3 apply_dna_patch.py --out ./Qwen3.5-0.8B-SGNE-CC
apply_dna_patch.py --verify-only)In each of the two modified feed-forward matrices of MIT-p50c (Takehi, Teoh, Zhang, Khattab, MIT CSAIL 2026) — layer indices 15 and 19, zero-based — the first singular mode accounts for 99.9995% and 99.9991% of update energy, while individual column norms increase by up to 114.3%. Once random sampling and the presence penalty are removed (greedy decoding T = 0, pp = 0), MIT-p50c's strict score on full MATH-500 drops from 259 of 500 (51.80%) to 25 of 500 (5.00%) with 95.00% cutoffs, and its accuracy falls by 2.2x–13.5x across all five suites; SGNE-CC outperforms MIT-p50c on all 5 of 5 benchmarks under greedy decoding.
SGNE-CC satisfies three verifiable weight properties:
mlp.down_proj columns in 15 feed-forward layers and 1,182 linear_attn.out_proj columns in 9 recurrent GatedDeltaNet layers; attention layers are unchanged.| Benchmark & Regime (PyTorch BF16, 32,768-token limit) | Qwen3.5-0.8B | MIT-p50c | SGNE-CC (this model) |
|---|---|---|---|
| Maximum column norm deviation (BF16) | 0.0000% | +114.2614% | 0.1625% |
MATH-500 (500) — Greedy (T = 0, pp = 0) |
0.00% (100% cutoffs) | 5.00% (25/500, 95.0% cutoffs) | 43.00% (215/500, 18.4% cutoffs, 8.6k tok, p = 3.24 × 10⁻⁵¹) |
MATH-500 (500) — Card (T = 0.6, pp = 1.5) |
7.80% (39/500, 92.2% cutoffs) | 51.80% (259/500, 21.1k tok, 28.2% cutoffs) | 47.00% (235/500, 5.3k tok, 1.6% cutoffs) (vs. vLLM s0 48.2%: p = 0.67) |
MGSM-240 (6 languages) — Greedy (T = 0, pp = 0) |
2.08% (5/240, 32.0k tok) | 21.25% (51/240, 24.0k tok) | 38.33% (92/240, 8.2k tok, p = 1.67 × 10⁻⁶) |
MGSM-240 (6 languages) — Card (T = 0.6, pp = 1.5) |
25.83% (62/240) | 57.08% (137/240, 8.6k tok) | 41.67% (100/240, 3.1k tok, p = 9.04 × 10⁻⁵) |
BBH-160 (clean_bbh) — Greedy (T = 0, pp = 0) |
0.00% (0/160, 100% cutoffs) | 3.75% (6/160, 94.4% cutoffs) | 41.25% (66/160, 7.2k tok, p = 2.73 × 10⁻¹⁷) |
BBH-160 (clean_bbh) — Card (T = 0.6, pp = 1.5) |
1.88% (3/160) | 50.62% (81/160, 16.9k tok) | 41.25% (66/160, 2.4k tok, 0.6% cutoffs, p = 0.058) |
HumanEval-164 (pass@1) — Greedy (T = 0, pp = 0) |
5.49% (9/164, 29.9k tok) | 14.02% (23/164, 26.3k tok) | 32.32% (53/164 — study high, 5.1k tok, p = 5.30 × 10⁻⁶) |
HumanEval-164 (pass@1) — Card (T = 0.6, pp = 1.5) |
15.85% (26/164) | 31.10% (51/164, 8.8k tok) | 28.05% (46/164, 1.1k tok, 0.0% cutoffs, p = 0.473) |
MultiSpeech-64 (8 languages) — Greedy (T = 0, pp = 0) |
3.12% (2/64) | 26.56% (17/64, med 32.8k) | 35.94% (23/64, med 2.2k tok, p = 0.210) |
MultiSpeech-64 (8 languages) — Card (T = 0.6, pp = 1.5) |
67.19% (43/64) | 90.62% (58/64, 2.5k tok) | 89.06% (57/64, med 1.9k tok, p = 1.000) |
All 5 suites (1,128 tasks) — Greedy T = 0 |
1.42% (16/1,128, 32.1k) | 10.82% (122/1,128, 28.5k) | 39.80% (449/1,128, 8.0k tok, 18.8% cutoffs) |
All 5 suites (1,128 tasks) — Card pp = 1.5 |
15.34% (173/1,128, 25.6k) | 51.95% (586/1,128, 15.0k) | 44.68% (504/1,128, 3.7k tok, 1.4% cutoffs) |
The MGSM p-value uses 40 six-language problem blocks; p-values are nominal, and a non-significant difference does not establish model equivalence. Each task has one direct PyTorch run (MIT's external vLLM reference on full MATH-500 under model-card settings gives 241/500 = 48.2% on run 0 and 48.1% four-run mean); hardware assignment notes appear in the GitHub repository.
T = 0, pp = 0) SGNE-CC wins on all five benchmarks. On MultiSpeech-64, the automated filter checks completion and repetition thresholds rather than factual accuracy (SGNE-CC retains repetition failures in German and isolated entity substitutions).LICENSE); any assembled checkpoint inherits the Apache-2.0 terms of Qwen/Qwen3.5-0.8B (see NOTICE and THIRD_PARTY_LICENSES.md)..npz patch, 38 trace files, and CPU scripts (52 files under SHA-256).model.safetensors (1,746,942,632 bytes): d0b745ce6e48ff0b2b22f9dadf229ba246afabebd9ae8308f07a4cd1824d0324sgne_cc_dna_delta.npz (3,277,552 bytes): 4e4d36e032869bc10f56e2a2033a52d3f2774321cf867ef6f74274a7c0d1566dmodel.safetensors (1,746,942,600 bytes): 04b1c301231dd422b8860db31311ab2721511346a32cb1e079c4c4e5f1fe4696model.safetensors 1.7 GBsgne_cc_dna_delta.npz 3 MBQwen/Qwen3.5-0.8B