Ternary Bonsai 2 27B — MTP bundles (GGUF)

PrismML's Ternary Bonsai 2 27B (a 27B dense model with end-to-end ternary weights, Qwen3.8-27B hybrid backbone, 262K context, Apache-2.0) with a NextN / MTP draft head bundled in, so the model self-speculates — no second file, no external drafter.

The published PrismML GGUFs carry no NextN layer. These bundles add the upstream Qwen/Qwen3.8-27B MTP head (its mtp.* tensors, exported by upstream llama.cpp convert_hf_to_gguf.py --mtp --outtype q8_0) as one extra layer; every other key and tensor is the PrismML file, byte for byte. The head stays in the normal basis (it is not among the Hadamard-rotated tensors); its token-embedding lookup on the rotated table gets the inverse transform in the engine (PrismML's 518ad108).

The vision projector is not duplicated here: use PrismML's Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf (600 MiB) from the base repo with --mmproj.

Running

These files need an engine that runs ternary weights and a bundled NextN head on a qwen35 hybrid: opencoti's llamafile (from the c11 cut) does; PrismML's own llama.cpp fork runs the ternary weights but not the bundled head.

# opencoti llamafile — NextN self-speculation; the draft depth for the ternary
# file types is a built-in table row (depth 2), no flag needed.
./opencoti-llamafile --server -m Ternary-Bonsai-2-27B-PQ2_0-MTP.gguf \
    -ngl 99 -fa on --spec-type draft-mtp

# with image input
./opencoti-llamafile --server -m Ternary-Bonsai-2-27B-PQ2_0-MTP.gguf \
    --mmproj Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf -ngl 99 -fa on --spec-type draft-mtp

The opencoti setup wizard lists this repo under the Ternary Bonsai 2 family and offers the projector as an optional download (it is loaded at the first image request, nothing is reserved for it at boot).

What MTP buys (greedy, 256-token answers, 12 chat prompts, medians)

Speculation preserves the model's distribution, not the argmax at near-ties: with greedy sampling an answer can differ from the unspeculated one at a position where the top-2 tokens are within a few hundredths of a nat (the verify batch rounds differently from single-token decode). No degeneration was measured (distinct 4-gram ratio identical on and off).

Reproducing the bundle

1. Export the head: convert_hf_to_gguf.py --mtp --outtype q8_0 Qwen/Qwen3.8-27B (upstream llama.cpp) → mtp-Qwen3.8-27B-Q8_0.gguf (c9e5c641de0c9896641c07a400ed92075024fe2c623549d427b1a6e44075bc0a).
2. Bundle it into the PrismML GGUF with opencoti's bonsai2-mtp-bundle.py: the head becomes layer block_count, nextn_predict_layers = 1; the script refuses unless every shared qwen35.* hyperparameter and the vocab agree and no tensor collides. The result is byte-identical across machines (the two sha256 above).

License

Apache-2.0, as the base model and the Qwen3.8-27B head. Credits: PrismML for Bonsai 2, the Qwen team for Qwen3.8-27B.