Published by TokForge for on-device inference.

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

TokForge-Bonsai-2-27B-GGUF

Prism ML's Bonsai 2 27B ternary model, packaged for on-device Android inference in TokForge, with the settings we measured to run best on phones.

Created using Bonsai by Prism ML. All credit for the model, the ternary training and the GGUF packings goes to Prism ML; the base model is Qwen3.8-27B by Alibaba Cloud. The PQ2_0 and PTQ1_0 files here are byte-identical to the upstream release; the MTP file adds one block from the base model to the PTQ1_0 file (see below). What TokForge adds is the Android runtime (Vulkan and CPU kernels for these packings) and the tested phone settings below.

Which file to use

• Have a Snapdragon 8 Elite Gen 5 or 8 Gen 3 phone with 16 GB of RAM? Pick PQ2_0. It is the larger file, but it runs on the GPU.
• The original Snapdragon 8 Elite (SM8750) is not on the GPU list yet and runs the CPU route.
• Otherwise pick PTQ1_0 for the smaller download and the CPU route.

TokForge chooses the route and the settings for you when it loads either file. You need TokForge 1.3.4 or newer.

MTP head: Ternary-Bonsai-2-27B-PTQ1_0-MTP.gguf

• What it is. The PTQ1_0 file above with one multi-token prediction (MTP) block added. On the CPU route TokForge can use it to draft a few tokens ahead and have the full model check them in one pass. A drafted token is kept only when the full model would have produced it, so the answers come from the full model.
• Same model. All 851 tensors of Ternary-Bonsai-2-27B-PTQ1_0.gguf are in this file byte for byte, at the same offsets: the whole 5,935,527,936-byte tensor data block has the same sha256 (976fbb7218f52a5f239cf36c0bb0e146300c7db1786a6e88e71fadb920d36263). Removing the head again gives back the upstream PTQ1_0 file exactly (sha256 53107f530aa52eb00912263ab1ee29bd199261c87cd7b4ad4ca1318c1fe33ee3).
• Header changes. qwen35.block_count 64 to 65, qwen35.nextn_predict_layers = 1, and tokforge.mtp_graft.* provenance strings. The prism.hadamard.* keys, the tokenizer and every other key are unchanged.
• Where the head comes from. Prism ML's releases carry no MTP tensors (GGUF PQ2_0, PTQ1_0 and F16, and the MLX build). The head is the 15 mtp.* tensors of the base model, Qwen/Qwen3.8-27B at commit 1d4bf0f2ff (model-00018-of-00018.safetensors), written as block 64 in the llama.cpp layout: weight matrices in Q8_0, norms in F32. The head reads Bonsai's own token embedding and output layer and carries no copy of either (451,319,808 bytes added). It was not retrained on Bonsai.
• Host test (desktop CPU, the TokForge engine's own MTP loop, greedy, 3 tokens drafted per round, 64 tokens, 20 prompts): share of drafted tokens the full model accepted: code 71%, advice 46%, chat 38%, documents (RAG) 84%. Greedy output with and without the head was identical on 20 of 20 prompts.
• Speed depends on the phone. TokForge measures each phone with and without the head and uses MTP only where it is faster; elsewhere this file runs like the plain PTQ1_0 file. The head adds about 0.45 GB to the download, and with MTP running the app uses more memory than with the plain file (the head plus extra working state). MTP needs the CPU route.

Phone notes

• Phone support:
  • The Snapdragon 8 Gen 3 (SM8650) uses the same GPU route, but this file was not separately benchmarked there.
  • Xiaomi phones on HyperOS cap an app at about 6 GiB of memory and closed both files in our tests.
  • 12 GB phones are untested.

Settings TokForge uses

These are applied automatically; you do not need to set them by hand. They are also in manifest.json for other runtimes.

PQ2_0 on the GPU (Snapdragon 8 Elite Gen 5 / 8 Gen 3, non-Samsung)

PTQ1_0 on the CPU

• Vulkan stays available for PTQ1_0 as an explicit choice, but it is slower today: the fast PTQ integer-dot GPU kernel is not yet correct on Adreno, so the GPU route falls back to a correct but slow path.
• Temperature and the other sampler settings are your own choice. TokForge's own tests use temperature 0 with seed 42 for repeatability; this is not a chat recommendation.

Using it in TokForge

1. Install TokForge 1.3.4 or newer.
2. Open Models, then Browse, and search for TokForge-Bonsai-2-27B, or download a file from this page and import it.
3. Load the model. TokForge picks the GPU or CPU route and the settings above.

Files

The PQ2_0 and PTQ1_0 files are byte-identical to upstream prism-ml/Ternary-Bonsai-2-27B-gguf at revision 6ed5e12bf84b7a63069882c91dd9e9218647d17b. The upstream repo also carries the F16 weights and the vision projector (mmproj) files, which TokForge does not use yet.

Credits

• Ternary Bonsai 2 27B GGUF by Prism ML (Hugging Face). Used under the Apache-2.0 license. The PQ2_0 and PTQ1_0 GGUF files, LICENSE and NOTICE.txt are byte-identical to it; the MTP file carries every PTQ1_0 tensor unchanged. Created using Bonsai by Prism ML. See their whitepaper and demo repository.
• Qwen3.8-27B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
• The MTP head in Ternary-Bonsai-2-27B-PTQ1_0-MTP.gguf is the multi-token prediction (MTP) layer of Qwen3.8-27B by the Qwen team (Alibaba Cloud) (Hugging Face, commit 1d4bf0f2ff), with its weight matrices converted to Q8_0 by TokForge. Used under the Apache-2.0 license.
• Ternary kernels from Prism ML's llama.cpp fork; TokForge ported them to Android.

License

Apache-2.0, inherited from Prism ML's release and the Qwen base model. LICENSE and NOTICE.txt are included unchanged. Per Prism ML's notice: created using Bonsai by Prism ML.

Community

• Website: tokforge.ai
• Discord: Join our Discord
• Benchmarks: Phone speed leaderboard
• Prism ML: prismml.com and their Discord