Ternary-Bonsai-2-27B-Abliterated (PQ2_0)

🚧 PREVIEW RELEASE (v0.1) — feedback wanted. This is an early preview of a quant-native abliteration of the Bonsai 2 pack. The numbers below are from my own small harness, so I would rather have the community poke at it than call it finished. Please open a Discussion in the Community tab with anything you find — good or bad.

Status: preview (v0.1) — not a final release. A refusal-reduced build of prism-ml/Ternary-Bonsai-2-27B-gguf (PQ2_0, 2.13 bpw, 7.21 GB). The pack is edited in its own deployed quantization format: there is no BF16 dequantization anywhere in the pipeline and no re-quantization.

Method (high level)

The official pack's stored 2-bit codes were updated directly, in place, so the shipped artifact is the edited pack: same geometry, same size, same kernel path, no runtime hook, no adapter. A full-precision refusal-free reference of the same base family was used only to define what to change; that change was then written onto the pack's existing quantization lattice with an unbiased code-rounding schedule, so that the expected weight movement equals the intended edit even though each individual code can only move in whole lattice steps. The edit was amplitude-swept and selected on measured behaviour.

Direction extraction, the reference construction and the rounding schedule are intentionally not published here.

Measured behaviour

Strict labeler, greedy decoding, PrismML fork runtime. "refusal" counts both plain refusals and refusal-then-pivot answers; "comply" is substantive compliance.

Capability is unchanged within measurement noise on every probe: the executed math/coding/tool/agentic battery is identical item-for-item, GSM8K is 27 vs 28 of 30 and SST-2 is 29 vs 29 of 30 (greedy). The only behavioural difference the harness sees is the one that was intended: on the same harmful probe the parent refuses with a pivot while this build answers substantively.

Feedback I am looking for

• Does the edit hold up on other prompt sets / languages / framings you care about, or does refusal come back?
• Regressions: coding, math, long-context, tool use, multi-turn agentic behaviour — anything that looks broken or degraded compared to the parent pack.
• xhigh vs medium: medium is the recommended mode here; reports about xhigh (the template default) are especially useful, including instances of empty answers.
• Anything that looks like lattice damage: repeated text, language mixing, degraded fluency.
• If it works for you: your hardware, runtime/flags and sampling settings, so the preview notes can carry a compatibility list.

Reproduce-it-yourself numbers are more useful than impressions — the labeler I used, the instruments, and the exact prompts are all things I can share in the discussion.

Usage

The chat template defaults to reasoning_effort=xhigh; this build is tuned and measured at medium, so ask for it explicitly:

llama-server -m Ternary-Bonsai-2-27B-Abliterated-PQ2_0.gguf \
  -ngl 999 -fa on -c 32768 --jinja --temp 1.0 --top-p 0.95 --top-k 20 \
  --chat-template-kwargs '{"reasoning_effort": "medium"}'

Any OpenAI-compatible client works against http://host:port/v1/chat/completions.

Caveats

• Safety-reduced model. Refusal is largely removed in medium mode. Intended for research and uncensored local use.
• xhigh (the template default) is the weaker mode. Refusal is still down to 1/40 there, but 16/40 answers came back empty because the long xhigh thinking exhausted the 2500-token budget in our harness; raise max_tokens, or use medium.
• The capability battery is small (10 math, 20 coding, 15 tool, 5 two-step agentic) and this is a single-turn gate; multi-turn robustness was not measured.
• Only the 400 edited tensors differ from the parent; integrity was verified byte-wise (size identical, no non-PQ2_0 tensor touched). metrics.json in this repo holds the raw aggregate counts.

日本語メモ（プレビュー版・フィードバック募集）

prism-ml/Ternary-Bonsai-2-27B の PQ2_0 パックを、配布フォーマットのまま（BF16 へ 戻さず、再量子化もせず）2-bit コードを直接書き換えて拒否を除去したものです。ファイル サイズ・テンソル形状・カーネル経路は親パックと完全に同一で、編集対象外の 451 テンソルは バイト単位で同一です。thinking は medium を明示指定してください（既定の xhigh では 拒否除去が弱く、回答が空になる場合があります）。

これは v0.1 のプレビューです。 上の表は私の小さな検証ハーネスの結果なので、皆さんの 環境・プロンプト・言語で試した結果をぜひ Community タブの Discussion に書いてください。 特に「拒否が戻る例」「能力劣化」「multi-turn での崩れ」「xhigh で空回答になる例」を歓迎 します。手法の詳細は、検証がもう少し進んだ段階で共有します。