Ternary-Bonsai-2-27B-Crystal

Successor: Ternary-Bonsai-2-27B-ConstantKV — the same weights and crystallized heads with an exact store instead of the index: 1.18 GiB of attention state at every context length, +0.0084 nat per token on unseen books at 32k tokens, 40/40 pass keys, and 32k tokens on a 16 GB Mac mini.

Ternary Bonsai 2 27B with crystallized attention: the 16 full-attention layers of the Qwen3.8-27B hybrid keep a fixed state (an exact window of 1,024 tokens plus a fitted decay carrier) and a 4-bit index of the past, instead of a key-value cache in bf16. The weights are the unchanged Prism ML pack; they are carried on a .causal graph (the monorail) that binds every tensor, authenticates every file, and records the model's routes so that known prefixes are travelled instead of recomputed.

The first model of the constant-state line. Crystallized Attention is the precursor of the Holographic Causal Graph Machine: it replaced the growing cache of a 27B hybrid by a fixed state without training the model, and the same idea, carried through the whole network, became Qwen2.5-1.5B-Instruct-HCGM — a model compiled into a causal graph with a constant state of 14.4 MiB, 470 tokens/s on the Neural Engine of a Mac mini, and reusable answers as .mrail tracks.

Original against Crystal

Same weights, same tokenizer, same prompts; only the attention state of the 16 full-attention layers differs.

_{Memory of the 16 full-attention layers (a) and decode time per token (b), original against Crystal. Decode measured in MLX on the CUDA backend (NVIDIA RTX PRO 6000 Blackwell, batch 1, whole model).}

The ternary weights themselves cost 0.28 nat per token against bf16 Qwen3.8-27B; the attention adds 1–2 % of that with the index.

What is different

• Attention without a bf16 cache. Per head: exact attention over the last 1,024 tokens, and beyond it a crystallized tail — five exponential modes with half-lives from a fixed catalogue (1 … 3,000 tokens) times a rank-8 query–key term, kept as a recurrent state. The 32 far tokens with the highest 4-bit score are read exactly from the index (int8 keys, int4 values, Hadamard-rotated), and the carrier's share of them is subtracted exactly.
• Constant-memory mode. With --top0 only the window and the state remain: 469 MiB for all 16 layers at any length, the fastest decode from 32k tokens on (1.5× the native cache at 64k).
• Monorail. Every request records its route through the model in the .causal graph (a route point every 1,024 tokens: recurrent states, crystal state, index rows). A request that shares a prefix with an earlier one travels the recorded route and computes only the rest — bit-identical to computing everything. In an editor-like workload (the same file completed six times while it grows) this is 5.5× faster. Confirmed continuations are recorded too and proposed as drafts that the model checks in one block: a repeated request runs 2.3–2.9× faster with 4–6 tokens per forward pass, and every emitted token is the argmax of the model's own logits. --routes-gb bounds the stored route points (least recently used first).
• Authenticated pack. bundle.json lists every shard and every file with SHA-256; the weights are read through their graph coordinates. A modified byte anywhere is rejected at load time.

_{From the paper: (a) the crystallized head — native logits inside the band, beyond it a key bias times a few exponential modes; (b) the fixed-state decoder of one key-value group — exact window, one recurrent state per mode, and the recall path. The released model uses a window of 1,024 tokens and five modes from a fixed catalogue.}

_{One read of one query head: exact frontier, the compressed state, and the top-k tokens read exactly from the index; the index subtracts the carrier's share of those tokens so nothing is counted twice.}

_{Qwen3.8-27B at 32,768 tokens: cost per position without and with the exact top-32 index (a), and state size against cost (b). The index brings the far context back to ~0.01 nat per token at a few MiB per layer; the native cache at 32k is 128 MiB per layer.}

Quickstart

hf download tfwnotops/Ternary-Bonsai-2-27B-Crystal --local-dir crystal-bonsai
pip install -r crystal-bonsai/crystal_runtime/requirements.txt      # Metal on Apple Silicon, CUDA 12.9 on Linux
pip install --find-links crystal-bonsai/crystal_runtime crystal_runtime
python crystal-bonsai/crystal_runtime/quickstart_crystal.py --pack crystal-bonsai \
    --prompt "Write a Python function that returns the n-th Fibonacci number iteratively." \
    --routes ~/.crystal/routes --bank ~/.crystal/bank

--file code.py --complete completes a file; --top0 switches to the constant-memory mode; --native runs the same pack with the native cache for comparison. Generation is greedy.

Checked from this repository on an RTX 3090 host (PyTorch CUDA image, Python 3.11): monorail verified (2,519 bindings), model ready in 20 s. The first request compiles the CUDA kernels once; the same request again replays its route: 128 tokens in 5.3 s, 6.74 tokens per forward, identical text.

Files

Papers

• Crystallized Attention (paper/Foss_2026_Crystallized_Attention.pdf) — fixed-state carriers and exact index correction for the full-attention heads of a 27B hybrid; the method of this model and its full evaluation.
• The constant-state line continues in the three papers of Qwen2.5-1.5B-Instruct-HCGM: The Holographic Causal Graph Machine (the compiler and holographic Ward recall), No GPU, No KV Cache (the Neural Engine runtime) and Compute Once (answers as reusable .mrail tracks).

Citation

@misc{foss2026crystal,
  author = {Foss, David Tom},
  title  = {Crystallized Attention: Fixed-State Carriers and Exact Index Correction for the Full-Attention Heads of a 27B Hybrid Language Model},
  year   = {2026},
  doi    = {10.13140/RG.2.2.21299.18725}
}
@misc{foss2026hcgm,
  author = {Foss, David Tom},
  title  = {The Holographic Causal Graph Machine: Compiling a Pretrained Transformer into a Constant-State Model without Training},
  year   = {2026},
  doi    = {10.13140/RG.2.2.28206.68167}
}
@misc{foss2026ane,
  author = {Foss, David Tom},
  title  = {No GPU, No KV Cache: A Constant-State Language Model on the Neural Engine of a Mac mini},
  year   = {2026},
  doi    = {10.13140/RG.2.2.31562.12484}
}
@misc{foss2026computeonce,
  author = {Foss, David Tom},
  title  = {Compute Once: Constant-State Language Models Make Answers Reusable and Agents Cheap},
  year   = {2026},
  doi    = {10.13140/RG.2.2.18140.35202}
}

License

Weights: Prism ML, Ternary Bonsai 2 27B (Apache License 2.0), derived from Qwen3.8-27B (Apache License 2.0); crystal parameters and monorail graph: Apache License 2.0 (LICENSE, NOTICE). The compiled runtime in crystal_runtime/: research license (crystal_runtime/LICENSE; free for non-commercial research and evaluation, commercial use by license). The .causal format and the live graph come from Immer. Author: David Tom Foss (d.foss@ieee.org, ORCID 0009-0004-0289-7154).