Qwen3.8-27B DFlash 2 drafter, ternary, for Ternary Bonsai 2

A small "draft" model that makes Ternary Bonsai 2 27B generate text faster, without changing what it says. It guesses the next several tokens in one go, and Bonsai checks the guesses; every token you get is one Bonsai would have produced anyway.

This is the drafter from naklitechie/Qwen3.8-27B-DFlash2-ternary-bonsai2, with its weights converted to the same three-valued (ternary) format as Bonsai itself and then fine-tuned to recover the accuracy that conversion costs. The result is about a third the size, and it runs on the AMD Ryzen AI NPU through OpenFlowLM.

Credit

This model is built on other people's work, and the first three did the hard parts:

• DFlash, the block-diffusion drafting method: Chen, Liang, Liu and colleagues (paper, z-lab).
• DFlash 2 and the Qwen3.8-27B drafter (z-lab/Qwen3.8-27B-DFlash2, from Inco AI): the architecture and the starting weights.
• naklitechie: fine-tuned that drafter on Ternary Bonsai 2's own output. Our starting point.
• PrismML: Ternary Bonsai 2 itself, and the rotation trick (below) we reuse. The drafter borrows Bonsai's token embedding and output head, so it only works with Bonsai 2.

All upstream models are Apache-2.0, and so is this one.

What we changed

1. Ternary weights. Every projection and feed-forward matrix is stored as values of -1, 0 or +1 with a scale per 128 weights, after the same Hadamard rotation Bonsai uses. That is OpenFlowLM's t2 format, so the NPU reads it with the kernels it already has for Bonsai. Converting without retraining lost about 12% of the tokens produced per pass at three drafts; the fine-tune cuts that to 1-4% (2-7% at seven drafts).
2. Fine-tune. About 1.7 million positions of text, each labelled with what Bonsai itself would say next (including its top-32 probabilities, so the drafter learns Bonsai's uncertainty, not only its favourite). Prompts came from UltraChat, GSM8K, ARC, HumanEval, OpenAssistant, CNN/DailyMail, a Python code-instruction set, and excerpts of Python's standard library and package documentation. The answers are Bonsai's, not the datasets'. Source datasets and their declared licences: UltraChat 200k (MIT), GSM8K (MIT), ARC (CC BY-SA 4.0), OpenAssistant oasst2 (Apache-2.0), CNN/DailyMail (Apache-2.0), HumanEval (MIT), and python_code_instructions_18k_alpaca (no licence declared; derived from sahil2801/code_instructions_120k, also undeclared). Only their prompts were used. About $7 of rented GPU time.

How good is it

Tokens produced per Bonsai verification pass, over 20 prompts (chat, code, document questions, reasoning), measured in llama.cpp. Higher is better.

(Each cell is chat / code / docs / reasoning.) Code is the weakest class, because it was the smallest share of the training data.

On an AMD Ryzen AI 9 HX 370 running OpenFlowLM, decoding is 1.56x (chat), 1.75x (code), 1.61x (documents) and 1.61x (reasoning) faster than Bonsai alone, with output identical token for token. These are early numbers from one machine and one set of prompts.

Using it

This model runs only in OpenFlowLM. No other engine supports it, including llama.cpp and PrismML's llama.cpp fork, and none is planned. If you want it in another engine, you will have to add the support yourself.

It needs an OpenFlowLM build with DFlash 2 support (not released yet) and a Bonsai 2 install: oflm add links the drafter to the model and oflm serve --spec turns speculation on. Usage will be documented in OpenFlowLM when it lands.

Limits

• Works only with Ternary Bonsai 2 27B; the drafter reads its hidden states and borrows its embedding and output head.
• Trained on English-heavy text; other languages and very long contexts are untested.
• The training data was generated by Bonsai 2 and so inherits its quirks.
• Greedy decoding only in OpenFlowLM today; sampling falls back to normal decoding.

Files

• model.q4nx: the drafter in OpenFlowLM's container format (1.4 GB).
• Qwen3.8-27B-DFlash2-Bonsai2-ternary.gguf: the drafter's weights (746 MB) in the same layout as PrismML's Ternary Bonsai 2 GGUF: every ternary matrix is a PQ2_0 tensor (2-bit codes, one fp16 scale per 128 weights) in a 1024-block Hadamard basis, described by the prism.hadamard.* keys (all signs +1). model.q4nx holds the same codes and scales. Only OpenFlowLM reads either file.
• config.json: drafter configuration.
• LICENSE, NOTICE: Apache-2.0 and attributions.