How to Run Ternary Bonsai 2 4B Without Refusals Locally

A 2.4 MB adapter, not a checkpoint. The 0.95 GB ternary pack stays byte-identical and the refusal direction is removed inside the compute graph, at a strength you set per request. The measurements behind this card are public.

• Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.
• Load it with --lora-scaled Ternary-Bonsai-2-4B-Uncensored-LoRA.gguf:1. Scale 1 takes held-out refusals to 0%, scale 0 switches it off, so one process serves both.
• It needs PrismML's llama.cpp fork - stock llama.cpp and Ollama cannot open the base pack at all.
• Research-grade: MMLU is unchanged, but the residual leak along the direction is about 1e-4, not the 1e-6 of the 27B adapter.

---

Ternary-Bonsai-2-4B-Uncensored-LoRA-GGUF

A rank-1 LoRA adapter that removes a refusal direction from prism-ml/Ternary-Bonsai-2-4B-gguf at inference time, built and measured by AtomicChat. The base pack stays byte-identical; nothing is requantized. The direction was estimated on the ternary pack itself.

All sizes

The 27B was measured with a 64-token reply budget and 500 MMLU questions on fork release prism-b10709-9a9394a; the smaller sizes with a 256-token budget and 300 questions on prism-b10777-e3de715. Compare base with adapter inside a row, not rows with each other. All four adapters and their measurements are in one collection, Ternary Bonsai 2 Uncensored.

Get the files

hf download prism-ml/Ternary-Bonsai-2-4B-gguf Ternary-Bonsai-2-4B-PTQ1_0.gguf --local-dir .
hf download AtomicChat/Ternary-Bonsai-2-4B-Uncensored-LoRA-GGUF Ternary-Bonsai-2-4B-Uncensored-LoRA.gguf --local-dir .

Run it with a build of PrismML-Eng/llama.cpp (verified on prism-b10777-e3de715):

llama-cli -m Ternary-Bonsai-2-4B-PTQ1_0.gguf --lora-scaled Ternary-Bonsai-2-4B-Uncensored-LoRA.gguf:1

Or serve it, with the strength as a per-request knob:

llama-server -m Ternary-Bonsai-2-4B-PTQ1_0.gguf -ngl 99 --jinja \
    --lora Ternary-Bonsai-2-4B-Uncensored-LoRA.gguf --lora-init-without-apply

curl localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "..."}],
  "lora": [{"id": 0, "scale": 1}]
}'

Older fork builds, such as prism-b10709 that the 27B adapter was made on, refuse these packs with unsupported prism.hadamard.version: 2.

What the adapter touches

Why an adapter and not a baked model: the weights are ternary, so a rank-1 edit written back into the pack would mostly round away, while llama.cpp keeps a LoRA as two extra matmuls in the graph and never merges it. The 27B card has the numbers for that size.

How the direction was estimated

Difference of means of the residual stream at the last prompt token (chat template applied, thinking off), 416 harmful prompts from mlabonne/harmful_behaviors against 416 harmless prompts from mlabonne/harmless_alpaca, sampling seed 0. The component along the mean harmless activation is projected out, because the naive difference makes the model refuse ordinary questions. The held-out test splits were not used to estimate the direction, but the row was picked by separation on them, and the evaluation below runs on the same splits, so it is not fully independent of that choice.

Evaluation

104 harmful + 104 harmless held-out prompts, greedy, 256-token budget, thinking off, same seed and system prompt everywhere. Refusal is a rule-based opening-phrase match, which is indicative, not a judge. An empty or degenerate reply is counted as damage, not as compliance.

Scale 1 already takes refusals to 0% on the held-out split; 1.5 adds nothing and produces 2 degenerate replies.

MMLU, 300 questions sampled with seed 0 from cais/mmlu, answer forced to a single letter by a root ::= [A-D] grammar, thinking off.

68.7 → 69.0: no change at this sample size.

Limitations

• Measured on PTQ1_0 only. PQ2_0, Q2_0 and the MTP and DFlash builds were not run with the adapter.
• Text only. Image and video inputs through the mmproj were not tested with the adapter.
• The residual leak along the direction at scale 1 is about 1e-4, measured inside the running fork, against about 1e-6 for the 27B adapter: the Hadamard basis of these packs is not yet matched exactly. Refusals are gone and harmless prompts are unaffected, but treat the adapter as research-grade. The leak figures are not in the metrics repo.
• The refusal metric matches opening phrases only. A reply that answers with a disclaimer counts as compliance.
• Capability was checked with MMLU alone. No math, code or agentic benchmark.
• The direction comes from one prompt-pair source at one token position. Transfer to other phrasings and languages is unmeasured.

Responsible use

This adapter removes a learned refusal direction, so the model will answer requests the published model declines. It is an inference-control and research artifact. Nothing about it makes the resulting output safe, correct or lawful, and any deployment needs its own access controls and policy enforcement.

Credits

Direction estimation, adapter build and evaluation by AtomicChat.

Base model Ternary Bonsai 2 4B by prism-ml, Apache-2.0, redistributed here in no form - this repository ships only the adapter.

The refusal-direction method follows Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024). The rank-1 LoRA formulation for these packs follows OrcaRouter's OrcaBonsai-27B-Uncensored, Apache-2.0.