Website  •  Learn more  •  Quantizations  •  Evaluation  •  How to use  •  License

Swift Bonsai 2 GGUF

Swift Bonsai 2 is UkisAI's reasoning-efficient derivative of Prism ML's Ternary Bonsai 2 27B. It uses 39.8% fewer thinking tokens while scoring 0.19% higher than the base.

Both the 1-bit and 2-bit quantizations are available in this repository; each file is a plain Bonsai 2 pack.

Quantizations

Each file is a complete model with the Swift correction merged into the ternary weights: no adapter file, patch, or extra flag is needed, and the files are the same size as the base Bonsai 2 packs. PTQ1_0 remains the earlier Swift release; PQ2_0 now contains updated merged weights.

Training approach

We built Swift by identifying reasoning-marker tokens that, in our analysis, trigger overthinking in the model's reasoning rollouts. We then fine-tuned the model by penalizing usage of those tokens while it reasons.

Swift produces shorter reasoning traces while keeping accuracy in line with the base model.

Evaluation

GPQA and C-Eval compare base Ternary Bonsai 2 27B with the historical Swift runtime correction under the saved protocols. IFBench and AIME compare base PQ2_0 with the current Swift PQ2_0 file. See the note below. Scores are the percentage of correct responses across all scored repetitions: three complete GPQA-Diamond runs and five runs each of C-Eval, IFBench, and AIME 2025. Token reductions are relative to the base model.

Token statistics measure thinking tokens, except AIME, where they measure the full completion. IFBench uses the official loose scorer on the full 1,500 responses. An up arrow means the current 2-bit model used more tokens. Token changes are not a direct measurement of latency or cost changes.

Benchmark methodology and reproduction settings

Sampling: temperature 1.0, top-p 0.95, top-k 20, min-p 0, repetition penalty 1, presence penalty 0, and no additional inference-time logit penalty.

• C-Eval covers the complete validation split, not the hidden test set. One duplicated prompt is cached per seed; all question IDs are scored and weighted separately.
• AIME scoring: the current base and Swift PQ2_0 files used identical corrected prompts (30 questions × 5 seeds, output cap 81,920) and the same archived Math-Verify 0.9.0 scorer, which checks the final response and falls back to reasoning. That scorer gives 92.00% base / 93.33% Swift. Final-answer-only scoring gives 90.67% / 90.67%. The AIME token columns report full completion tokens.
• IFBench reports official loose scoring for the current base and Swift PQ2_0 files (300 questions × 5 seeds). The current run completed all 1,500 responses without request errors; prompt and sampler settings were spot-checked against the base run.
• Model selection: GPQA informed the choice of training settings and is not an untouched holdout. Named benchmark exclusions and exact-match checks were performed; a comprehensive near-overlap audit was not completed.
• GPQA and C-Eval are the original measured base/Swift runtime results. IFBench and AIME were refreshed on the current PQ2_0 file against base PQ2_0. PTQ1_0 remains the earlier merged release. Reproducibility can vary with runtime builds and hardware.
• Small score differences are not established improvements. Exact aggregates, truncation counts, and confidence intervals where computed are available in benchmark_results.json.

Research status and limitations

This is an experimental research release. In our internal evaluations and practical testing, benchmark scores did not consistently translate into reliable general-purpose behavior. Instruction following, tool use, and open-ended coding or agent tasks remain uneven. These observations concern the specific models, runtimes, and tests we used; they do not establish a general conclusion about ternary models.

We see ternary quantization as a promising direction for making larger models more accessible. Further advances in training, low-bit adaptation, and inference support may make the approach increasingly useful. This release shares a research result and its current limitations, rather than presenting a production-ready model.

How to use

Download and run

These files use Prism's PQ2_0 and PTQ1_0 ternary packs and run on the PrismML-Eng/llama.cpp fork (tested at revision 1a07bfa5f). Stock llama.cpp does not support these tensor types. No patch, adapter file, or extra flag is needed.

Linux prerequisites: Git, CMake, a C++ compiler, and the NVIDIA CUDA toolkit.

git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release -j
hf download ukisai/Swift-Bonsai-2-GGUF Swift-Bonsai-2-PQ2_0.gguf --local-dir .
./build/bin/llama-server -m Swift-Bonsai-2-PQ2_0.gguf --host 127.0.0.1 --port 8080 \
  -ngl 99 -c 32768 --flash-attn on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --repeat-penalty 1.0

To run 1-bit / PTQ1_0 instead, download Swift-Bonsai-2-PTQ1_0.gguf and pass it to -m.

The command above serves a 32,768-token context on 127.0.0.1:8080. Use -c 98304 for the longer evaluation caps if memory permits. Runtime memory includes model weights and context/state caches; download size is not total VRAM usage.

OpenAI-compatible API

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "bonsai-2-swift",
    "messages": [{"role": "user", "content": "Explain the difference between correlation and causation."}],
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 20,
    "min_p": 0.0,
    "max_tokens": 4096,
    "chat_template_kwargs": {"enable_thinking": true, "reasoning_effort": "xhigh"}
  }'

The launcher flags above are the intended model settings. Custom integrations should preserve them.

License and attribution

Created using Bonsai by Prism ML. This is an independent UkisAI release, not an official Prism ML release or endorsement.

The model is distributed under Apache-2.0. The upstream notice and attributions are retained.

Citation

@misc{swift-bonsai-2-gguf,
  title  = {Swift Bonsai 2 GGUF},
  author = {UkisAI},
  year   = {2026},
  url    = {https://huggingface.co/ukisai/Swift-Bonsai-2-GGUF}
}