Boole-1.7B-GGUF

A small and strong 1-bit LLM

Safetensors  ·  GGUF  ·  💬 On-Device Chat  ·  Blog  ·  Code

Boole-1.7B is a 1-bit language model derived from Qwen3-1.7B, intended for English-language inference in non-thinking mode. This GGUF export runs with llama.cpp on CPU or CUDA.

The figure uses the 247.1 MB packed source weights; this GGUF is 292.1 MB because it stores the embedding and output matrices separately. See the main model card for evaluation settings.

Download

Quick start

The boole-demo repository builds the pinned llama.cpp runtime.

git clone https://github.com/PrecisionNumerics/boole-demo.git && cd boole-demo
hf download precision-numerics/Boole-1.7B-GGUF Boole-1.7B-Q1_0.gguf --local-dir model
bash scripts/setup_llama_cpp.sh
third_party/llama.cpp/build/bin/llama-server \
    -m model/Boole-1.7B-Q1_0.gguf -ngl 99 -c 4096 \
    --reasoning off

For CPU, build with GGML_CUDA=0 bash scripts/setup_llama_cpp.sh and use -ngl 0. Open the server URL printed in the terminal.

License

Apache 2.0. NOTICE includes upstream Qwen attribution.