Saina Helm 2 Omni 0.8B — GGUF

llama.cpp build of run-saina/saina-helm-2-omni-0.8b, a decision model that reads text, images and audio and answers typed questions with probabilities. No text is generated.

F16 backbone (1.5 GB GGUF), the F16 vision tower as a llama.cpp multimodal projector (mmproj-…, 205 MB), and Helm's FP32 head. A small runner, omni-hidden, built against llama.cpp and its mtmd library, returns the hidden states the head reads; score.py builds the prompt and applies the head.

Built from source revision aea9ac53af6fdfac7e5da48d0cbf3213e18a7b40 (2026-10-20).

How it works

Helm 2 Omni answers every question from one pass over a shared prompt: the images first (each audio clip is turned into a log-mel spectrogram image and read the same way), then the context and all the questions, then one readout slot per question. At each slot, the hidden state goes through Helm's head: a linear layer over the answer codes plus 16 small residual experts, of which the top 2 are routed in. Softmax over the offered codes gives single-choice probabilities; a sigmoid per code gives multi-label ones.

score.py takes the prompt, the image sizes and the spectrograms from the saina package itself (saina[tokenize,media], no PyTorch), so they match what the model was trained on exactly. Images are resized to the model's patch grid before they reach llama.cpp, so llama.cpp does not resize them again. The head isn't linear and reads several positions, so it ships as experts.safetensors and runs in NumPy. Stock llama-server, llama-mtmd-cli and Ollama can load these files but can't produce Helm's answers; use the runner below.

Parity. Against the official saina runtime (PyTorch FP32), probabilities stay within 0.00048 on every test request: text only, a screenshot, a chess board, a chart, an audio clip, and an image with an audio clip together, including a multi-label question (details in parity.json). Don't quantize without re-checking parity: the head reads hidden states directly.

Usage

Build the runner against llama.cpp at the commit it was tested with (its C API changes between versions):

hf download run-saina/saina-helm-2-omni-0.8b-GGUF --local-dir omni-gguf
git clone https://github.com/ggml-org/llama.cpp && git -C llama.cpp checkout abeada335e2e78bd3fe63febafab7e900ce75810
cmake -S llama.cpp -B llama.cpp/build -DLLAMA_BUILD_TESTS=OFF   # add -DGGML_CUDA=ON or -DGGML_METAL=ON for a GPU
cmake --build llama.cpp/build --target llama mtmd -j 8
c++ -std=c++17 -O3 -include cstring omni-gguf/omni-hidden.cpp \
    -Illama.cpp/include -Illama.cpp/ggml/include -Illama.cpp/tools/mtmd -Lllama.cpp/build/bin \
    -Wl,-rpath,"$PWD/llama.cpp/build/bin" -lllama -lmtmd -lggml -lggml-base -o omni-hidden
pip install numpy safetensors "saina[tokenize,media]>=0.4.0"

import sys; sys.path.insert(0, 'omni-gguf')
from score import Helm

helm = Helm('omni-gguf', runner='./omni-hidden')

helm.ask('Inbound call to the bank: a short recording of English speech. '
         'The caller asks to raise their daily transfer limit.',
         [{'question': 'Is this speech synthesized or voice-converted by a computer (a spoof)?',
           'choices': ['No', 'Yes']}],
         audios=['call.wav'])            # WAV, FLAC, MP3, Ogg or WebM, up to 10 s each, up to 4 clips

helm.ask('A chess puzzle from an online game.',
         [{'question': 'Black to move. Which move is best?', 'choices': ['Rf7', 'h6', 'Rxf1', 'a5']}],
         images=['board.png'])           # PNG, JPEG or WebP, up to 8 images
# -> one probability list per question, in the order of `choices`

Helm(..., threads=, n_ctx=, gpu_layers=) sets the runner's CPU threads, context size (default 32,768 tokens, images and spectrograms included) and layers offloaded to a GPU (needs a GPU build of llama.cpp). Parity was measured on CPU.

Audio questions work best when the context says what the clip is ("a short recording of English speech") and the options name the sounds plainly; see the model card for intended use, limits and evaluation.

Files