Phocinae-Largha-150M-v1 — ONNX (fp16)

fp16 ONNX export of Phocinae/Phocinae-Largha-150M-v1: a 144.3M decision model (mmBERT-small encoder + typed-decisions head) that answers noul (yes/no), choice (pick-one) and score (2–10) questions in a single forward pass. Runs on ONNX Runtime — no PyTorch, no phocinae-server.

fp16 is the model's native precision (the upstream checkpoint ships as fp16 safetensors), so this is a format conversion, not a lossy compression.

Quick start

hf download ahmadw/Phocinae-Largha-150M-v1-ONNX --local-dir ./largha-onnx
pip install -r ./largha-onnx/requirements.txt

import sys; sys.path.insert(0, "./largha-onnx")
from largha_onnx import OnnxEngine

eng = OnnxEngine("./largha-onnx")   # providers=["CUDAExecutionProvider", ...] also accepted
answers, confidence, action, usage = eng.run(
    "The agent restarted nginx after checking the logs and the health endpoint is green.",
    [{"id": "ok",   "type": "noul", "threshold": 0.65},
     {"id": "act",  "type": "choice", "options": ["allow", "ask", "deny"]},
     {"id": "risk", "type": "score"}],
)
# answers    -> {'ok': False, 'act': 0, 'risk': 3}
# confidence -> {'ok': 0.6678, 'act': 0.409, 'risk': 0.1633}   (PyTorch reference: 0.668 / 0.409 / 0.1633)

OnnxEngine.run is a drop-in for phocinae.engine.Engine.run: same request shape, same return tuple, perm=True for 4-order option permutation averaging, with_scores=True for per-option probabilities. Calibration temperatures from rl_agent_config.json are applied.

Graph interface

Outputs: logits [batch, options] (pre-temperature option scores) and act_probs [batch, 2] (escalate head). Sequence construction lives in largha_onnx/sequence.py.

Parity with the reference PyTorch runtime

Checked against phocinae-server 0.1.6 on CPU (scripts/check_onnx_engine.py):

• tokenizer: 13/13 edge-case strings produce identical ids (CJK, emoji, special tokens, whitespace);
• decisions: identical answers and token counts on all 14 runs (7 cases × perm on/off, incl. a 1024-token state, a 40-option choice, injected <mask>), max probability drift 0.0003;
• the fp32 graph this was converted from matches PyTorch to ~1e-6 on logits.

These are spot checks, not a re-run of the full typed-decisions benchmark; see the base model card for benchmark numbers, which were measured on the PyTorch runtime.

Why no int8

Dynamic int8 quantization was tested and rejected: quantizing the encoder MatMuls changed 6–7 of 14 test decisions (probability drift up to 0.48). The model's decisions are often close margins sharpened by calibration temperatures < 1, so this small, deep encoder does not tolerate per-tensor activation quantization. Embedding-only int8 was near-lossless but no smaller than fp16.

Limitations

All limitations of the base model apply (see its model card): it is a decision aid, not a security product; tool routing confuses semantically close tool names; benchmark claims carry the base card's training-overlap disclosure.

License

Apache-2.0, same as the base model. Base encoder: jhu-clsp/mmBERT-small (MIT). See NOTICE.