Ternary Bonsai 2 · 4B — NInfer
The NInfer build of Ternary Bonsai 2 4B:
PrismML's ternary quantisation of Qwen3.5-4B, repacked into a single-file .ninfer container.
The original release is GGUF (llama.cpp). This repository carries the same weights in the NInfer container format so the NInfer engine can load them directly.
Model
Ternary Bonsai 2 4B is a ternary version of Qwen3.5-4B: weights take only {−1, 0, +1}, with one FP16 scale shared per 128 input channels and a block-wise Hadamard rotation basis — roughly 2 bits per weight.
| Base model | Qwen3.5-4B |
| Text layers | 32 |
| Hidden size | 2560 |
| Intermediate size | 9216 |
| Context length | 262,144 |
| Vocabulary | 248,320 |
| Attention | Hybrid: 1 full-attention layer per 4 (layers 3, 7, 11, 15, 19, 23, 27, 31), the rest linear |
| Query / KV heads | 16 / 4 (head_dim 256) |
| Modality | Text (this container carries no vision tower) |
| Base licence | Apache-2.0 |
Files
| File | Size | SHA-256 |
|---|---|---|
Ternary-Bonsai-2-4B-PQ2_0.ninfer |
1,307,514,880 B | afd399a9384a137bb6a7bcdc7157e8756cca3ea7619db061f8e60b8ed24112d9 |
SHA256SUMS holds the full digests; .conversion.json records the conversion recipe.
Usage
Load with the NInfer engine:
ninfer-serve Ternary-Bonsai-2-4B-PQ2_0.ninfer \
--host 127.0.0.1 --port 8094 \
--max-context 8192 --kv-dtype bf16
It then serves an OpenAI-compatible API at /v1/chat/completions.
Provenance
Qwen3.5-4B base model Qwen/Qwen3.5-4B Apache-2.0
Ternary-Bonsai-2-4B ternary checkpoint prism-ml/Ternary-Bonsai-2-4B-gguf see its LICENSE
this repository NInfer container repacked here
The ternary weights and the rotation basis come from PrismML. This repository only converts the container format: the two-bit code words are moved byte for byte, never dequantised and never requantised, and no value is changed.
Converted with the toolchain in
ninfer-bonsai2-converter (converter-v3/), which carries the recipe,
both verifier suites and the full reproduction trail.
Use
Research and local inference. Not validated for production or safety-critical use.