Cerno 9B Q4_0 (GGUF)

The on-device build of Cerno 9B (v1.1 weights) · 4-bit · llama.cpp · one forward pass · calibrated probabilities

Cerno 27B · Cerno 9B · Cerno 4B · 9B GGUF · 4B GGUF · Inference code

What it is

The 4-bit llama.cpp build of Cerno 9B (made from the v1.1 weights) for phones, laptops and anything llama.cpp runs on. Same interface as the FP8 model: a state and a set of typed questions, a probability for every option of every question in one forward pass, no text generation. Made with quantisation-aware training to llama.cpp's Q4_0 grid, so it keeps the quality of the full-precision model:

The run through llama.cpp used the maintainers' kit at commit 9eb2dbe, llama-server on one H200 with the public engine (evals/decision_index/engine_llama.py, greedy read-out), every request answered, nothing truncated. Run-to-run noise on the public index is about 0.1-0.2 points.

Format

• Body: every linear weight in Q4_0 (block-32 symmetric 4-bit, the format the Snapdragon NPU, the Adreno GPU and the ARM CPU paths of llama.cpp all accelerate). Head and embeddings: Q8_0. Exactly the forward the model was trained for; do not re-quantise it to another mix.
• File: cerno-9b-v1.1-q4_0.gguf (6.07 GB), SHA-256 in SHA256SUMS.
• Temperature: the shipped calibration temperature (1.539) lives in decider_config.json next to the FP8 weights (apertixai/cerno-9b); the engine reads it from there.

Run it

# 1. serve the file (any llama.cpp build; on Android see the notes below)
llama-server -m cerno-9b-v1.1-q4_0.gguf --host 127.0.0.1 --port 8080 -c 32768 -b 8192 -ub 2048 -fa on -ngl 99

# 2. score the Decision Index sample or the whole suite with the public engine
git clone https://github.com/apertixsk/cerno-inference && cd cerno-inference
pip install -r requirements.txt && export PYTHONPATH=$PWD:$PWD/third_party/decider
python -m decision_index pipeline --edition 0.3 --engine evals.decision_index.engine_llama:LlamaGGUFEngine \
    --option model=apertixai/cerno-9b --option gguf=cerno-9b-v1.1-q4_0.gguf --option base_url=http://127.0.0.1:8080 \
    --out runs/cerno-9b-q4_0

model= points at the FP8 repo only for the tokenizer and decider_config.json; no FP8 weights are downloaded. Programmatic use is the same as for the FP8 model (engine(state, questions)), see the inference repository.

Android / Snapdragon: build llama.cpp with its Snapdragon preset (Hexagon NPU + Adreno OpenCL + CPU in one binary, see docs/backend/snapdragon/README.md upstream), push the file, start llama-server on the phone with --devices HTP0:0 (NPU), --devices GPUOpenCL (GPU) or -ngl 0 (CPU), forward the port with adb forward tcp:8080 tcp:8080, and run the same command from the laptop. The workload is one forward pass per question, so prompt-processing throughput is the whole latency.

Family

Licence and credits

Apache-2.0, like the FP8 weights it was made from (base model Qwen3.5-9B-Base, Apache-2.0). Quantisation format by the llama.cpp project. Jev and System One are TypeSafe AI's names; this project is not affiliated with TypeSafe AI or with the Decision Index maintainers.