Bonsai 2 27B CRACK, compact PTQ1_0 for MLX
This is a text-only MLX safetensors repack of the PTQ1_0 GGUF from
dealignai/Bonsai-2-27B-1bit-CRACK-GGUF.
The 402 ternary matrices retain their original 28-byte-per-128-weight PTQ1_0
blocks; they are not converted to MLX affine 2-bit weights. A separate
mlx-kquant Metal operator multiplies those packed blocks during inference.
The model.safetensors file is 5,994,496,877 bytes (5.99 decimal GB); it is
47,847,949 bytes larger than the source GGUF. The package also includes a
12 MB tokenizer and small runtime files.
This is a packaging and runtime integration of existing CRACK weights and existing packed Metal operators, not a new compression algorithm or model. Conversion source and verifier: Obscyra-app/bonsai-crack-ptq1-mlx. There is already a CRACK 1.75-bit JANG MLX release with a different packed codec and vMLX runtime. This package preserves the PTQ1_0 representation of the GGUF and exposes a small local text API.
Requirements
- Apple Silicon Mac, macOS 26.2 or newer, Python 3.10–3.14.
mlx-kquant==0.4.13(which pinsmlx==0.32.1),mlx-lm==0.31.3, NumPy, tokenizers and Jinja2; install withpip install -r requirements.txt.- Enough unified memory for the 6 GB weights, activations and KV cache. Tested on MacBook Air M3 with 16 GB unified memory.
Run
Download this complete repository with the official Hugging Face CLI, then run from the downloaded directory:
hf download Nurymanau/Bonsai-2-27B-CRACK-PTQ1-MLX --local-dir bonsai-crack-ptq1-mlx
cd bonsai-crack-ptq1-mlx
pip install -r requirements.txt
python3 quickstart.py 'Сколько будет 2+2? Ответь только числом.'
python3 server.py --host 127.0.0.1 --port 8091
The server accepts text-only, non-streaming, deterministic
POST /v1/chat/completions requests and supports GET /v1/models.
It defaults to loopback access. Example:
curl http://127.0.0.1:8091/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"bonsai-crack-ptq1-mlx","messages":[{"role":"user","content":"2+2?"}],"temperature":0,"max_tokens":16}'
This custom PTQ1_0/Hadamard schema is not loadable by stock mlx_lm.load().
The included quickstart.py and server.py are the supported entry points.
Vision inputs, streaming, sampling, LoRA and long-context quality are not
validated in this text package.
Reproduction and checks
Source GGUF revision: 9665897ee63152226ecd64a1036d14292397186f.
Source file SHA256:
dcca61238e280432c4ce2d4c6c1d61214cbcb8a5d93ed98dea20d8bcef385a42.
This package's model.safetensors SHA256:
e6815083fbd8d0945840f66ed0299be89dcaef4094b6fccc4c5cfdadb27dbb9b.
An independent verifier checked all 402 packed matrices byte-for-byte against
the source GGUF after the required model row reordering.
See build-manifest.json and the project builder
bonsai_crack_compact_build.py for the exact repacking path.
On the tested M3/16 GB Mac, fresh package load took 2.19 s; four short
greedy text prompts produced the same visible answers as the source GGUF:
4, Paris, 6, and Привет. This small smoke check does not establish
general capability or full numerical equivalence. The measured decode rate
on the three-token Привет response was about 4.0 tokens/s, versus roughly
5.1–5.4 tokens/s from the PrismML GGUF Metal server on these short prompts.
Credits and license
Model: Prism ML Bonsai 2,
derived from Qwen3.8-27B. CRACK weights: dealignai.
Packed Metal operator: asher/mlx-kquant,
MIT licensed, installed as a dependency. The Prism ML license and notice are
included. Created using Bonsai by Prism ML. The weights are distributed under
the source model's Apache 2.0 license. This repack does not claim ownership
of the base model, CRACK modification, or mlx-kquant kernels.