Saina Helm 2 0.8B — GGUF

llama.cpp build of run-saina/saina-helm-2-0.8b, a decision model that answers typed questions with probabilities. No text is generated.

F16 backbone (1.5 GB GGUF) plus Helm's FP32 head. A small runner, helm2-hidden, built against llama.cpp, returns the hidden states the head reads; score.py builds the prompt and applies the head.

How it works

Helm 2 answers every question from one pass over a shared prompt: the context and all the questions, followed by one readout slot per question. At each slot, the hidden state goes through Helm's head: a linear layer over the answer codes plus 16 small residual experts, of which the top 2 are routed in. Softmax over the offered codes gives single-choice probabilities; a sigmoid per code gives multi-label ones.

The head isn't linear and reads several positions, so it can't be folded into the GGUF's output layer. It ships as experts.safetensors, and score.py runs it in NumPy. Stock llama-server, llama-cli and Ollama can load the GGUF but can't produce Helm's answers; use the runner below.

Parity. Against the official saina runtime (PyTorch FP32), probabilities stay within 0.0008 across the test requests, including multi-question requests, a 40-option question, multi-label questions and a 4,427-token input (details in parity.json). For v1.0, a Q8_0 build drifted by up to 0.036 and was not shipped. Don't quantize further without re-checking parity: the head reads hidden states directly.

Usage

Build the runner against llama.cpp at the commit it was tested with (its C API changes between versions):

hf download run-saina/saina-helm-2-0.8b-GGUF --local-dir saina-helm-2-gguf
git clone https://github.com/ggml-org/llama.cpp && git -C llama.cpp checkout abeada335e2e78bd3fe63febafab7e900ce75810
cmake -S llama.cpp -B llama.cpp/build -DLLAMA_BUILD_TESTS=OFF   # add -DGGML_CUDA=ON or -DGGML_METAL=ON for a GPU
cmake --build llama.cpp/build --target llama -j 8
c++ -std=c++17 -O3 saina-helm-2-gguf/helm2-hidden.cpp \
    -Illama.cpp/include -Illama.cpp/ggml/include -Lllama.cpp/build/bin \
    -Wl,-rpath,"$PWD/llama.cpp/build/bin" -lllama -o helm2-hidden
pip install numpy safetensors transformers

import sys; sys.path.insert(0, 'saina-helm-2-gguf')
from score import Helm

helm = Helm('saina-helm-2-gguf', 'saina-helm-2-gguf/saina-helm-2-0.8b-f16.gguf', runner='./helm2-hidden')

helm.ask('Please refund me, the package never arrived and nobody answers the phone.', [
    {'question': 'Which team should handle this?',
     'choices': ['Billing', 'Technical support', 'Shipping and delivery']},
    {'question': 'Which apply?', 'mode': 'multi_label',
     'choices': ['Refund request', 'Delivery problem', 'Praise', 'Support unreachable']},
])
# -> [[p_billing, p_technical, p_shipping],            single_label: sums to 1
#     [p_refund, p_delivery, p_praise, p_unreachable]]  multi_label: independent

helm.score('Can we move our call to Thursday?', 'Is a reply needed?', ['Yes', 'No'])

Helm(..., threads=, n_ctx=, gpu_layers=) sets the runner's CPU threads, context size (default 32,768 tokens; the model accepts up to 262,144) and layers offloaded to a GPU (needs a GPU build of llama.cpp). Parity was measured on CPU; a GPU build should be re-checked before you rely on it.

score.py builds the exact prompt Helm 2 was trained on. If you build prompts yourself, keep them byte-for-byte identical, or the probabilities drift. Questions take 2 to 588 options. Longer input than the context size is rejected, never truncated.

The richer request format (yes/no, rating, decision mode with thresholds and explicit fallbacks) lives in the saina package and the hosted API at saina.run; this build returns the raw probabilities those are built on.

Intended use and limitations

Same as the main model, whose card has the evaluation, training data and limitations in full. In short: use it for recoverable or reviewed decisions (intent routing, triage, tool shortlisting, review checklists), validate on your own labelled data, and set thresholds there, because probabilities are not calibrated. English only; not for safety filtering or for decisions about people.

Previous model: saina-helm-0.8b-GGUF. Apple silicon: saina-helm-2-0.8b-MLX.

License

Apache-2.0, as for the main model. Training data sources keep their own terms; see the main model card.