Ternary Bonsai 2 2B — WebGPU with images

Created using Bonsai by Prism ML. This is a modified ONNX packaging of Prism's Ternary Bonsai 2 2B, for local text and image inference in WebBrain.

The ternary language weights and Hadamard transforms come unchanged from danyalmalik/Ternary-Bonsai-2-2B-ONNX-WebGPU. The original Prism FP16 vision tower is exported into the compatible onnx-community/Qwen3.5-2B-ONNX topology. All original vision tensors are mapped and checked after serialization; the MLX patch projection is transposed from its channel-last layout. Normalization and position tensors are widened to FP32 as required by the ONNX graph. No training or additional language quantization is performed. Processor sidecars select the compatible Qwen2 tokenizer class and a 1,048,576 pixel image ceiling for browser memory use.

Use Transformers.js AutoProcessor and AutoModelForImageTextToText, with device: 'webgpu' and dtype: {embed_tokens: 'q2f16', decoder_model_merged: 'q2f16', vision_encoder: 'fp16'}. The runtime must support Qwen3.5 hybrid linear/full attention, MatMulNBits with 2-bit weights, GatherBlockQuantized, and FP16 WebGPU shaders. Vision pixels and image embeddings enter the graph as FP32. KV and recurrent states are FP16.

Reproduce with scripts/models/bonsai/build_vision.py in the WebBrain repository. export-validation.json records pinned upstream revisions, complete vision tensor mappings, serialization hashes, and CPU graph checks. manifest.json records distributable file sizes and SHA-256 hashes. Browser and model-answer validation must be run separately; graph checks alone do not establish answer quality or support on every browser/GPU.

Color/shape fixtures check independent visual facts. Exact OCR is a separate optional check. The 2B model omitted or confused characters in four-character OCR fixtures, including at a 1,024-visual-token budget. Failed OCR reports are retained alongside successful pipeline checks when publishing validation evidence.

Apache 2.0; see LICENSE and NOTICE.txt. Original context capacity is 262,144 tokens; WebBrain applies a smaller browser context budget. Model weights and processor files are downloaded once; inference runs locally.

Parallel prefill

The upstream language graph computes every GatedDeltaNet prefill with an ONNX Scan over tokens. ONNX Runtime's WebGPU execution provider has no Scan kernel, so prompts were processed one token at a time through a CPU loop with a GPU round trip per token and layer (0.13 s per prompt token for 2B and 0.19 s for 4B on an RTX 5090; a faster GPU barely helps). This graph replaces each Scan with the chunked gated delta rule (128-token chunks; stable recursive block inversion inside a chunk; a parallel prefix scan across chunks), built only from ops the WebGPU EP implements, and returns logits for the last position only. Weights, quantization and the external data file are byte-identical to the previous revision; decode (one token at a time) uses the unchanged recurrent path.

chunked-prefill-validation.json records the rewrite, CPU comparisons with the original graph on real text (logit differences at fp16 rounding level, identical top-1 tokens, prefill lengths across chunk boundaries and a cached continuation) and WebGPU prefill timings against the original graph. Reproduce with scripts/models/bonsai/chunk_gdn_prefill.py in the WebBrain repository.

Hardware validation

The packaged WebBrain browser worker passed text generation, a native tool call and tool-result round trip, two independent image-only color/shape fixtures and image history with this graph on an NVIDIA RTX 5090 through Chromium WebGPU; see browser-validation.json. A cold reload with Hugging Face requests blocked passed the same checks; see offline-browser-validation.json. Greedy answers were identical to the previous graph's recorded answers except the blue image (identical for the first 228 characters); later tokens differ through FP16 rounding.

Wall time per check, previous graph → this graph (same harness and GPU):

These synthetic smoke checks do not establish general answer quality or compatibility with other GPUs.

Decisions (decision_hidden)

WebBrain can use this model to answer quick questions about a web page, such as "is the task done?", "which option fits?" or "how complete is this, 0–2?", without writing any text.

What changed in this file. The decoder graph has one extra output, decision_hidden: the 2,048 numbers the model computes just before it picks its next word (the final hidden state at the last position, FP16). Before, they stayed inside the graph. Nothing else changed: same weights, same chat answers. On the CPU the outputs are bit-for-bit identical to the previous graph. Apps that only read logits, such as Transformers.js generation, ignore the new output.

How WebBrain uses it.

1. It reads the page text and screenshots once.
2. For each possible answer it asks: "Suggested answer: X. Is that right? Yes or No?"
3. A small scorer (the probe: one weighted sum, 2,049 weights) combines how strongly the model leans to Yes with the 2,048 decision_hidden numbers into a confidence for each answer. The most confident answer wins.

The model itself is not retrained; only the probe is trained, on 288 practice questions from WebBrain's d1 decision set. The probe ships inside the WebBrain extension (src/chrome/src/offscreen/bonsai-decision-probe.js).

Results. 144 held-out questions on 72 synthetic screenshots, run in the WebBrain extension on an RTX 5090. "No probe" asks one multiple-choice question and reads which letter the model favors. d1 is WebBrain's dedicated decision model, webbrain-one/d1-browser-decision-fp16.

In short: the probe turns this chat model into a usable decision maker, and it gets the most answers right. d1 is still the safer check for "is it really done?" (fewer false "done" calls) and about three times faster.

Caveats. All questions come from one generator of synthetic, AI-labelled screens, and the probe was trained on the same kind of data. This does not prove quality on real websites. Measured on one GPU.

Details are in decision-validation.json. Reproduce the graph with scripts/models/bonsai/add_decision_hidden.py and the CPU check with scripts/models/bonsai/check_decision_hidden.py in the WebBrain repository. The same build also passed WebBrain's 12-task page-question check and its Settings image check. The timing tables earlier in this README were recorded before this output was added.

Observed OCR limitation

An earlier 448×448 / 576×384 check (196/216 visual tokens) identified both colors but misread K7M2 as KM2 and R4X9 as AX9. See low-resolution-ocr-validation.json. Exact text recognition is resolution-sensitive and can be wrong. Prism recommends a higher image-token budget for OCR; WebBrain caps each image at 1,048,576 pixels for browser memory use.

OCR reports in this repository were recorded with the previous (Scan) graph.