Sign up

com-kotobalabs/Ternary-Bonsai-2-27B-Refusal-Steering-GGUF

🤗 Hugging Face sourcetext-generationapache-2.01 MBGGUF✓ HF checksums availableupdated today
No torrent yet →

Ternary Bonsai 2 27B Refusal Steering (GGUF)

Experimental refusal-direction control vector for prism-ml/Ternary-Bonsai-2-27B-gguf.

This repository does not contain a fully uncensored model. It contains a 1.3 MB GGUF control vector and the reproducibility material used to test refusal steering. The held-out evaluation did not support a reliable uncensored claim, so this artifact is published under its measured scope.

Measured status

  • Base artifact: Ternary-Bonsai-2-27B-PTQ1_0.gguf
  • Base revision: 6ed5e12bf84b7a63069882c91dd9e9218647d17b
  • Runtime: PrismML-Eng llama.cpp fork at 9a9394a895b96003ca842a6041cb28ac49a108f7
  • Hardware: Apple M1 Max, 32 GB unified memory
  • Extraction: mean harmful-minus-benign activation at the final prompt token, normalized independently for layers 1–63
  • Recommended experimental scale: -2.0, layers 10–63
  • Base refusal check: 4/4 tested categories refused
  • At -2.0 one training prompt changed internal reasoning toward conditional compliance, but its final answer still refused
  • Held-out check at -2.0: 4/4 still showed refusal-oriented reasoning
  • At -3.0 over layers 10–63 output quality collapsed
  • At -3.0 over layers 20–40 benign arithmetic remained correct (17 x 23 = 391), but 4/4 held-out prompts still refused

These are small diagnostic runs, not a safety or capability benchmark.

Use

The base model requires Prism ML's llama.cpp fork.

git clone https://github.com/PrismML-Eng/llama.cpp
cmake -B llama.cpp/build
cmake --build llama.cpp/build -j --target llama-server

hf download prism-ml/Ternary-Bonsai-2-27B-gguf \
  Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir models

hf download com-kotobalabs/Ternary-Bonsai-2-27B-Refusal-Steering-GGUF \
  refusal-direction.gguf --local-dir models

llama.cpp/build/bin/llama-server \
  -m models/Ternary-Bonsai-2-27B-PTQ1_0.gguf \
  -ngl 99 -fa on \
  --control-vector-scaled models/refusal-direction.gguf:-2.0 \
  --control-vector-layer-range 10 63

Negative scale suppresses the measured harmful-minus-benign direction. Positive scale strongly amplifies it and is not useful for this purpose.

Reproduce

extract_refusal_vector.py loads the original ternary GGUF through Prism's MLX runtime, records every decoder-layer output for paired prompts, takes the difference at the final prompt token, averages each layer, normalizes it, and writes a llama.cpp-compatible control-vector GGUF.

The published Prism ML runtime rejected 96 BF16 auxiliary tensors in this GGUF. runtime-bf16.patch adds the GGUF reader's standard dequantize path for those tensors. Apply it to the runtime bundled with prism-ml/Ternary-Bonsai-2-27B-mlx-2bit before running the extractor.

The paired prompts are included. They are intentionally small and should be expanded before treating this as more than an experiment.

License and attribution

The base model is Apache-2.0. See LICENSE and NOTICE.txt. This repository does not redistribute the 5.95 GB base weights.