Ternary Bonsai 2 4B — WebGPU with images

Created using Bonsai by Prism ML. This is a modified ONNX packaging of Prism's Ternary Bonsai 2 4B, for local text and image inference in WebBrain.

The ternary language weights and Hadamard transforms come unchanged from danyalmalik/Ternary-Bonsai-2-4B-ONNX-WebGPU. The original Prism FP16 vision tower is exported into the compatible onnx-community/Qwen3.5-4B-ONNX topology. All original vision tensors are mapped and checked after serialization; the MLX patch projection is transposed from its channel-last layout. Normalization and position tensors are widened to FP32 as required by the ONNX graph. No training or additional language quantization is performed. Processor sidecars select the compatible Qwen2 tokenizer class and a 1,048,576 pixel image ceiling for browser memory use.

Use Transformers.js AutoProcessor and AutoModelForImageTextToText, with device: 'webgpu' and dtype: {embed_tokens: 'q2f16', decoder_model_merged: 'q2f16', vision_encoder: 'fp16'}. The runtime must support Qwen3.5 hybrid linear/full attention, MatMulNBits with 2-bit weights, GatherBlockQuantized, and FP16 WebGPU shaders. Vision pixels and image embeddings enter the graph as FP32. KV and recurrent states are FP16.

Reproduce with scripts/models/bonsai/build_vision.py in the WebBrain repository. export-validation.json records pinned upstream revisions, complete vision tensor mappings, serialization hashes, and CPU graph checks. manifest.json records distributable file sizes and SHA-256 hashes. Browser and model-answer validation must be run separately; graph checks alone do not establish answer quality or support on every browser/GPU.

Color/shape fixtures check independent visual facts. Exact OCR is a separate optional check. The 4B model omitted or confused characters in four-character OCR fixtures, including at a 1,024-visual-token budget. Failed OCR reports are retained alongside successful pipeline checks when publishing validation evidence.

Apache 2.0; see LICENSE and NOTICE.txt. Original context capacity is 262,144 tokens; WebBrain applies a smaller browser context budget. Model weights and processor files are downloaded once; inference runs locally.

Parallel prefill

The upstream language graph computes every GatedDeltaNet prefill with an ONNX Scan over tokens. ONNX Runtime's WebGPU execution provider has no Scan kernel, so prompts were processed one token at a time through a CPU loop with a GPU round trip per token and layer (0.13 s per prompt token for 2B and 0.19 s for 4B on an RTX 5090; a faster GPU barely helps). This graph replaces each Scan with the chunked gated delta rule (128-token chunks; stable recursive block inversion inside a chunk; a parallel prefix scan across chunks), built only from ops the WebGPU EP implements, and returns logits for the last position only. Weights, quantization and the external data file are byte-identical to the previous revision; decode (one token at a time) uses the unchanged recurrent path.

chunked-prefill-validation.json records the rewrite, CPU comparisons with the original graph on real text (logit differences at fp16 rounding level, identical top-1 tokens, prefill lengths across chunk boundaries and a cached continuation) and WebGPU prefill timings against the original graph. Reproduce with scripts/models/bonsai/chunk_gdn_prefill.py in the WebBrain repository.

Hardware validation

The packaged WebBrain browser worker passed text generation, a native tool call and tool-result round trip, two independent image-only color/shape fixtures and image history with this graph on an NVIDIA RTX 5090 through Chromium WebGPU; see browser-validation.json. A cold reload with Hugging Face requests blocked passed the same checks; see offline-browser-validation.json. Greedy answers were identical to the previous graph's recorded answers for every check.

Wall time per check, previous graph → this graph (same harness and GPU):

These synthetic smoke checks do not establish general answer quality or compatibility with other GPUs.