Ternary-Bonsai-2-27B-ConstantKV

Ternary Bonsai 2 27B whose attention state does not grow. Each of the 16 full-attention layers of the Qwen3.8-27B hybrid holds, from the first token on, 75.75 MiB: the exact band of the last 1,024 tokens, a crystallized carrier over every far token, and an exact store of 8,192 keys and values per key-value head, admitted by the sharpest look each token receives while it is still in the band. All 16 layers hold 1.18 GiB at every context length; the native cache of the same weights holds 3.2 GB at 32k tokens, 6.3 GB at 64k and 12.4 GB at 128k. At 32k tokens on eight unseen books the state costs +0.0084 nat per token and retrieves 40 of 40 pass keys, without training the model.

The whole model runs at 32k tokens on a Mac mini M4 with 16 GB: peak 12.2 GB, 155–157 ms per token at 4k, 16k and 32k tokens alike.

Successor of Ternary-Bonsai-2-27B-Crystal. The weights (Prism ML's pack) and the crystallized heads are byte-identical to Crystal, on the same .causal graph (2,519 tensor bindings). What changes is the state: Crystal read the far past from a 4-bit index that grows with the context (2.0 GB at 32k), or dropped it (--top0, 469 MiB, +0.116 nat). ConstantKV keeps a fixed exact store instead: the cost of the index at a constant size. The method and every number on this card are from the paper Scarcity as Method — Attention, Replaced: Exact Forms, Constant State, and Reusable Inference (doi:10.13140/RG.2.2.34347.40489, PDF in paper/).

Original, Crystal, ConstantKV

Same weights, same tokenizer, same prompts; only the attention state of the 16 full-attention layers differs.

The ternary weights themselves cost 0.28 nat per token against bf16 Qwen3.8-27B; the constant state adds a thirty-third of that. Two independent implementations of the store (PyTorch and this MLX runtime) agree to within ±0.0004 nat on every book:

_{Figure 1 of the paper, one real query: the last token of a 32k-token PG19 book, layer 27, the six heads of one key–value group. (a) Softmax attention reads every key and value of the past. (b) The constant state reads the band exactly (blue), the 8,192 stored tokens at their native logits (red), and every other far token through the carrier (grey). The red caps are the peaks of attention the carrier cannot carry; the store holds exactly these tokens. Right: the state of one full-attention layer, to scale.}

What the state is

• Band. The last 1,024 tokens, exact keys and values (16.0 MiB per layer, plus 2.9 MiB of their index rows).
• Carrier. Every token that leaves the band enters a crystallized carrier: a key bias times five exponential decays with half-lives from a fixed catalogue, with a rank-8 query–key term, kept as a recurrent state (1.1 MiB per layer). These forms were found by the exact formula engine Wirkung, not designed.
• Exact store. 8,192 tokens per key-value head keep their exact keys and values in FP16 and are read at their native logits; their share of the carrier is subtracted, so nothing is counted twice (55.8 MiB per layer). A token is admitted by the sharpest single attention weight it receives from inside the band; the first token is always kept. Admission by window mass at the same size retrieves 33 of 40 pass keys, the sharpest look 40 of 40.
• Exact below the store. Up to 1,024 + 8,192 = 9,216 tokens the state is the native attention of the model.
• A function of the tokens. The state is determined by the tokens read; so the greedy answer of a context is a property of that context and can be stored once.

_{Figure 5 of the paper: (a) one head, with the admission of a token that leaves the band by its sharpest look; (b) the constant state of one key–value group. In decoding, a token that leaves the band takes the slot of the weakest entry when its sharpest look is larger; two Metal kernels read the FP16 store in FP32.}

_{Figure 6 of the paper: (a) cost at 32k tokens against the size of the store, (b) cost by position in the book, (c) attention memory of the 16 full-attention layers against the context length. The carrier plus store stays flat; the native cache and the index grow.}

Answers that are not computed twice

The runtime keeps a .mrail map. Every answer is appended to it as a run; a request whose prompt the map has answered before is read from the file in 0.04–0.06 ms, without touching the model, and identical to decoding on the backend that wrote it (8 of 8 checked on CUDA; on the Mac, runs written by the Mac return identical). maps/coding_constantkv.mrail ships 1,385 answers of this runtime to coding requests (0.97 MB), written on CUDA: decoded again on a Mac, 5 of 8 checked runs are token-identical and the other three part at a near-tie, where the two best logits lie within 0.002–0.008. maps/coding_batch.mrail adds 256 more as drafts. With --draft on, the tree decoder drafts the next tokens from the maps and the running answer and verifies them in one pass; rejected branches are rolled back level-exactly without evaluating a layer twice (DeltaNet and crystal state are rewritten from recorded inputs). On 60 held-out coding requests one verified path accepts 1.18 tokens per pass without a map and 1.77 with the 1,641 runs, still rising with the size of the map. Every emitted token is the argmax of the model: drafts change the number of passes, never the output.

_{Figure 11 of the paper: (a) accepted tokens per pass on unseen coding requests against the size of the map, for this model (red, blue) and for the compiled 1.5B; (b) more paths verified per pass on the 27B. The released decoder verifies one path.}

Quickstart

hf download tfwnotops/Ternary-Bonsai-2-27B-ConstantKV --local-dir constantkv
pip install -r constantkv/crystal_runtime/requirements.txt      # Metal on Apple Silicon, CUDA 12.9 on Linux
pip install --find-links constantkv/crystal_runtime "crystal_runtime==2.0.0"
python constantkv/crystal_runtime/quickstart_constantkv.py --pack constantkv \
    --prompt "Write a Python function that returns the n-th Fibonacci number iteratively."

--file notes.txt --prompt "..." puts a long document in front of the question; --file code.py --complete completes a file; --native runs the same pack with the growing native cache for comparison; --store sets the store size. Your own map lives in ~/.constantkv/maps/own.mrail (started from the shipped one); asking the same thing twice returns the answer from the file. Generation is greedy.

On a 16 GB Mac close memory-heavy apps before a 32k-token context: the runtime itself peaks at 12.2 GB.

Checked from this repository, freshly downloaded into a new environment: on a Mac mini M4 (16 GB, Python 3.11 and 3.13) the monorail verifies every file, a known prompt returns from the map in 0.05 ms and a new one decodes at 8 tokens/s; on an NVIDIA RTX PRO 6000 Blackwell (Python 3.11, CUDA) eval/pg19_store.py reproduces the measurement behind the table above for book 6 at 32k tokens to the last digit (2.6237059 nat per token, 29 s), greedy decoding runs at 45.6 tokens/s, and the tree decoder with the shipped maps accepts 2.39 tokens per pass with output identical to greedy.

Files

Lineage

1. Ternary-Bonsai-2-27B-Crystal — crystallized attention: carrier plus exact index correction, the first constant-state model of the line.
2. Qwen2.5-1.5B-Instruct-HCGM — the same idea carried through a whole network: a transformer compiled into a causal graph with a 14.4 MiB state, 470 tokens/s on the Neural Engine of a Mac mini, answers as .mrail tracks.
3. Ternary-Bonsai-2-27B-ConstantKV — the 27B with a state that does not grow, at the quality of the index, on a 16 GB Mac.

Papers

• Scarcity as Method — Attention, Replaced: Exact Forms, Constant State, and Reusable Inference (doi:10.13140/RG.2.2.34347.40489, paper/Foss_2026_Scarcity_as_Method.pdf) — the program of work as one answer: the forms found by Wirkung, the laws that place a constant state, the exact store of this model, the compiled 1.5B, and inference that is reused.
• Crystallized Attention — fixed-state carriers and exact index correction for the full-attention heads of a 27B hybrid.
• The Holographic Causal Graph Machine, No GPU, No KV Cache and Compute Once — the compiler, the Neural Engine runtime and answers as reusable tracks.

Citation

@misc{foss2026scarcity,
  author = {Foss, David Tom},
  title  = {Scarcity as Method. Attention, Replaced: Exact Forms, Constant State, and Reusable Inference},
  year   = {2026},
  doi    = {10.13140/RG.2.2.34347.40489}
}
@misc{foss2026crystal,
  author = {Foss, David Tom},
  title  = {Crystallized Attention: Fixed-State Carriers and Exact Index Correction for the Full-Attention Heads of a 27B Hybrid Language Model},
  year   = {2026},
  doi    = {10.13140/RG.2.2.21299.18725}
}
@misc{foss2026computeonce,
  author = {Foss, David Tom},
  title  = {Compute Once: Constant-State Language Models Make Answers Reusable and Agents Cheap},
  year   = {2026},
  doi    = {10.13140/RG.2.2.18140.35202}
}

License

Weights: Prism ML, Ternary Bonsai 2 27B (Apache License 2.0), derived from Qwen3.8-27B (Apache License 2.0); crystal parameters, monorail graph and maps: Apache License 2.0 (LICENSE, NOTICE). The compiled runtime in crystal_runtime/: research license (crystal_runtime/LICENSE; free for non-commercial research and evaluation, commercial use by license). The evaluation scripts in eval/: Apache License 2.0. The .causal format and the live graph come from Immer. Author: David Tom Foss (d.foss@ieee.org, ORCID 0009-0004-0289-7154).