Saina Helm 2 Omni 0.8B — MLX

MLX build of run-saina/saina-helm-2-omni-0.8b, a decision model that reads text, images and audio and answers typed questions with probabilities. No text is generated.

FP16 backbone and vision tower with FP32 shifted RMSNorm weights (to avoid conversion rounding), loaded with stock mlx-vlm (its Qwen3.5 model, image merge and M-RoPE positions), plus Helm's FP32 head. score.py builds the prompt, reads the hidden state at each question's readout slot and applies the head.

Built from source revision aea9ac53af6fdfac7e5da48d0cbf3213e18a7b40 (2026-10-20).

How it works

Helm 2 Omni answers every question from one pass over a shared prompt: the images first (each audio clip is turned into a log-mel spectrogram image and read the same way), then the context and all the questions, then one readout slot per question. At each slot, the hidden state goes through Helm's head: a linear layer over the answer codes plus 16 small residual experts, of which the top 2 are routed in.

score.py takes the prompt, the image sizes and the spectrograms from the saina package itself (saina[tokenize,media], no PyTorch), and the pixels from the same Qwen image processor saina uses, so the inputs match what the model was trained on exactly. mlx_vlm.generate can load these weights but can't produce Helm's answers; use score.py.

Parity. Against the official saina runtime (PyTorch FP32), probabilities stay within 0.00155 on every test request in a four-case smoke suite: text only, a thumbnail image, a one-second audio clip, and the image and audio together (including multi-label output) (details in parity.json, measured with the FP16 weights computed in FP32 on a Linux CPU, mlx 0.32.4 and mlx-vlm 0.7.8; on Apple silicon the GPU computes in FP16, which can add a little more drift).

Usage

pip install "mlx-vlm>=0.7.8" "saina[tokenize,media]>=0.4.0"
hf download run-saina/saina-helm-2-omni-0.8b-MLX --local-dir omni-mlx

import sys; sys.path.insert(0, 'omni-mlx')
from score import Helm

helm = Helm('omni-mlx')

helm.ask('A chess puzzle from an online game.',
         [{'question': 'Black to move. Which move is best?', 'choices': ['Rf7', 'h6', 'Rxf1', 'a5']}],
         images=['board.png'])           # PNG, JPEG or WebP, up to 8 images

helm.ask('Inbound call to the bank: a short recording of English speech.',
         [{'question': 'Is this speech synthesized or voice-converted by a computer (a spoof)?',
           'choices': ['No', 'Yes']}],
         audios=['call.wav'])            # WAV, FLAC, MP3, Ogg or WebM, up to 10 s each, up to 4 clips
# -> one probability list per question, in the order of `choices`

Audio questions work best when the context says what the clip is and the options name the sounds plainly; see the model card for intended use, limits and evaluation.

Files