Ternary Bonsai 2 27B Abliterated v2 (PQ2_0 + MTP): GGUF mirror + RTX 3060 12 GB benchmarks

This repo has two parts:

1. A byte-identical mirror of Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP.gguf from BoldingBuilds/Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP-GGUF. We did not change the weights. For the model itself (how the abliteration and the thinking fix were made, and refusal/capability evals), read BoldingBuilds' card. It is the authoritative source.
2. Reproducible benchmarks on one RTX 3060 12 GB (bench/): a 30-task inbox-automation benchmark across 17 local model/runtime configs, a max-context search, speed/draft-acceptance numbers, the llama.cpp MTP patch we built with, and every raw result file.

Read this before you use the numbers: the only tests we ran on this exact file (abliterated v2 + MTP) were the speed and max-context tests. The 50-attempt quality table below has no row for this file. Its "B" rows are a different abliterated build without MTP, and its "J" rows are the non-abliterated base Bonsai 2 with MTP. See Honest notes.

Files

PQ2_0 is PrismML's ternary packing. Mainline llama.cpp, Ollama and LM Studio can't load it. You need PrismML's llama.cpp fork (see below).

How we ran it

Runtime: PrismML llama.cpp prism-b10685 + MTP patch

We built PrismML's llama.cpp fork from the prism-b10685 source and applied bench/scripts/0001-qwen35-mtp-hadamard-inverse.patch. Without the patch, loading a Bonsai MTP GGUF with --spec-type draft-mtp fails at startup with Hadamard-latent table 'token_embd.weight' is read without the inverse transform, because the MTP draft graph does its own embedding lookup and skips the inverse Hadamard transform. The patch adds that transform (about 14 lines in src/models/qwen35.cpp).

• The patch is the one published by decent-jawfish/bonsai-2-27b-mtp, byte for byte (SHA-256 c10f6222…ce23b0). Its git header names MFEC AI Lab as the author. BoldingBuilds ships a patch with the same code change.
• Build: bench/scripts/build_prismml_mtp.sh (CUDA 12.8, -DCMAKE_CUDA_ARCHITECTURES=86 for the RTX 3060, -DLLAMA_CURL=OFF, target llama-server).
• According to BoldingBuilds, PrismML tag prism-b10743-adfffbe or newer runs MTP files without the patch. We did not test that tag.

git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
git checkout prism-b10685-7dffb15        # the tag decent-jawfish lists for this patch
git apply /path/to/bench/scripts/0001-qwen35-mtp-hadamard-inverse.patch
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DLLAMA_CURL=OFF
cmake --build build --target llama-server -j

Launch command (RTX 3060 12 GB, fully on GPU)

This command is our production configuration with this file's largest safe context on the shared 3060 (see Max context):

./llama-server -m Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP.gguf \
  --host 0.0.0.0 --port 8801 \
  -ngl 999 -fa on -c 26624 -np 1 \
  -ctk q8_0 -ctv q8_0 -ctkd q8_0 -ctvd q8_0 -fit off \
  --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.05 \
  --chat-template-kwargs '{"reasoning_effort": "medium"}' \
  --spec-type draft-mtp --spec-draft-n-max 2

• --spec-type draft-mtp --spec-draft-n-max 2: the built-in MTP head (tensors blk.64.*) drafts 2 tokens per step.
• -ctk/-ctv q8_0 quantises the target model's KV cache and -ctkd/-ctvd q8_0 does the same for the draft's KV cache.
• -ngl 999 -fit off puts all layers on the GPU with no automatic fitting or CPU offload. -fa on turns on flash attention.
• The sampling flags are the server defaults (Bonsai's recommended temp 1.0 / top-k 20 / top-p 0.95, plus llama.cpp's min-p 0.05). The benchmark requests override them (see below). reasoning_effort is medium, which BoldingBuilds recommends for this build.

Results for this file (abliterated v2 + MTP)

Speed

Measured on the RTX 3060 12 GB at .54 while a vLLM embedder shared the GPU (~2.5 GB).

For comparison, the non-abliterated base Bonsai 2 MTP GGUF on the same card and binary gives 56.7–57.1 tok/s short, 33.8 tok/s at 90% of 28K, and 0.933 acceptance.

Max context (RTX 3060 12 GB)

Method (bench/scripts/ctx_test.py, raw data in bench/results/ctx_search_54.jsonl): launch llama-server fully on the GPU with the flags above, send a synthetic prompt that fills about 90% of -c, and check that it completes. A setting counts only if at least 300 MiB of VRAM stays free at peak. The 3060 was also running a vLLM embedder (2,548 MiB) during every run.

• Each extra 2,048 tokens of context costs about 46 MiB of VRAM (q8_0 KV, hybrid attention).
• This file is about 100 MB larger than the base MTP GGUF, so on the shared card it fits one 2K step less (26,624 vs 28,672).
• We did not measure this file on the dedicated 3060. The 77,824 figure is for the base Bonsai 2 MTP GGUF.

The 30-task inbox-automation benchmark

The question was which local model should handle email triage, extraction and drafting on our home hardware.

Tasks (prompts and graders in bench/code/): 30 unique tasks in three suites.

• easy (10): classify, priority, bill extraction, phishing, summarise, draft reply, batch classify, action items, Spanish, rule-driven tool call
• hard (10): final meeting time, conflicting bills, GST arithmetic, subtle phishing, rule engine, constrained Spanish reply, relative dates, strict schema, prompt injection, mixed-language batch
• ultra (10): GST + FX reconciliation, timezone/DST reschedule, phishing headers batch, 12-rule engine, action-items schema, injection via tool calls, Rioplatense constrained reply, invoice reconciliation, scheduling puzzle, multilingual taxonomy

Protocol

• Every config ran with thinking on (reasoning_effort: medium for Bonsai, max_tokens 3000) and thinking off (chat_template_kwargs.enable_thinking=false, max_tokens 800).
• Sampling per request: temperature 0.2, top_p 0.95, top_k 20, seed 42. The hard and ultra suites were run a second time with seed 7. Easy repeats at seed 42 gave identical outputs, so easy counts once. That makes 10 + 2×10 + 2×10 = 50 attempts per model per thinking mode.
• Strict grading: a truncated answer (hit max_tokens) counts as a fail. Requests were sent one at a time, and wall time was measured at the client.
• Winner: J54, the non-abliterated base Bonsai 2 27B PQ2_0 + MTP on the 3060. It scored 96% (48/50) with thinking on, 15.8 s median and 43.3 s max per task, ~52 tok/s.

Final comparison table (all 50 attempts, strict grading)

From bench/summary.md (updated 2026-09-28 03:35 NZDT). ".54" is the Linux RTX 3060 12 GB (shared with the embedder), "Win" is a dedicated RTX 3060 12 GB, and "Mac" is an Apple M5 with 24 GB.

Full per-task detail, the repeat runs, and caveats are in bench/summary.md. The bench README (bench/README.md) has the hardware list and the exact J launch command.

Honest notes

• No quality run for this file. This abliterated v2 MTP GGUF got the speed and max-context tests above, not the 50-attempt benchmark.
• "B" / "I" rows are Ternary-Bonsai-2-27B-Abliterated-PQ2_0.gguf: a different abliterated Bonsai 2 build with no MTP (B on the .54 3060, I on the Mac). They are not this file.
• "J54" / "JMac" rows are the non-abliterated base Bonsai 2 27B PQ2_0 with a grafted MTP head (Bonsai-2-27B-PQ2_0-MTP.gguf from decent-jawfish/bonsai-2-27b-mtp). They are also not this file.
• Speculative decoding checks every draft token against the target model, but batched verification can change floating-point reduction order. BoldingBuilds reports that this occasionally flips near-ties.
• The shared-GPU context limits depend on the ~2.5 GB embedder that was resident on that card. On a dedicated 12 GB card you get much more (see the 77,824 base-model figure).
• Hostnames and IPs are stripped from the code. bench/code/bench.py reads WIN_HOST, LINUX54_HOST and MAC_HOST from the environment.

Reproduce

cd bench/code
LINUX54_HOST=<host> python3 bench.py --model J --modes off,on --suite tasks_hard --out results_hard_J.jsonl
# --suite: tasks | tasks_hard | tasks_ultra ; --base overrides the endpoint URL
python3 summarize.py && python3 combined.py      # read results*.jsonl from the working dir
python3 ../scripts/ctx_test.py MODEL.gguf 26624 8899 ctx.jsonl   # max-context probe (edit BIN inside first)

Safety

This is an abliterated model: its refusal behaviour was removed, and it will comply with requests a stock model declines. As BoldingBuilds says, you are responsible for how you use it. Don't deploy it in a user-facing product without your own safety layer.

Credits

Every contributor below was checked against the upstream model cards.

• Prism ML: Ternary Bonsai 2 27B (prism-ml/Ternary-Bonsai-2-27B-gguf, Apache-2.0), the PQ2_0 format, and the PrismML-Eng/llama.cpp fork (MIT) we ran on. Created using Bonsai by Prism ML.
• Qwen team, Alibaba Cloud: Qwen/Qwen3.8-27B (Apache-2.0), the base Bonsai 2 is derived from and the source of the MTP head.
• Unsloth: unsloth/Qwen3.8-27B-GGUF. BoldingBuilds copied the MTP head tensors from it.
• BoldingBuilds: the abliteration (refusal edit plus the one-row end-of-thinking-token fix) and the MTP graft that make up this GGUF (source repo). BoldingBuilds' card presents the abliteration as their own work. Other abliterated builds (Hikari07jp, dealignai, OS-Software, Blackfrost) appear there only as comparisons.
• MTP recipe for Bonsai 2, as credited by BoldingBuilds: decent-jawfish and ProCreations, building on sudoingX/qwen38-mtp.
• MTP llama.cpp patch (bench/scripts/0001-qwen35-mtp-hadamard-inverse.patch): published by decent-jawfish. Its git header names MFEC AI Lab as the author.
• ggml-org/llama.cpp (MIT) and its contributors: the inference engine, GGUF, and the speculative-decoding (draft-mtp) support the PrismML fork builds on.
• Benchmark, context tests and this mirror: groxaxo (Facundo).

If you use Bonsai 2 27B, Prism ML asks you to cite:

@techreport{bonsai2_27b,
    title   = {Bonsai 2 27B: A 27B Ternary Reasoning Model},
    author  = {Prism ML},
    year    = {2026},
    month   = {September},
    url     = {https://prismml.com}
}

License

The weights are Apache-2.0, as are all upstream weights (Prism ML Bonsai 2, Qwen3.8-27B, Unsloth GGUF, BoldingBuilds' derivative). The weights are redistributed unmodified, with LICENSE and NOTICE included. The llama.cpp runtime is MIT-licensed; the included patch is a small change against it, published by decent-jawfish. The benchmark code and results in bench/ are released under Apache-2.0.

Not affiliated with or endorsed by Prism ML, Qwen/Alibaba Cloud, Unsloth, or BoldingBuilds.