laya base/English (ONNX FP32, single graph)
FP32 single-graph ONNX export of the base English
convaiinnovations/laya
checkpoint (base revision 55cf4c4e). 5-input marker form, logits +
act_logits, for ONNX Runtime. Same recipe and harness as the
typed-decisions sibling.
laya-base-fp32.onnx+laya-base-fp32.onnx.data(1.69 GB),tokenizer.json,rl_agent_config.jsonunchanged from upstream.- License: Apache-2.0. Config: max_len 512 / head_max_len 192.
- SHA-256 + toolchain pins: see
manifest.json.
Quality caveat (read before serving classifiers)
Per Ollaya's measurements the base checkpoint is near-chance zero-shot on typed decisions (0.357) — classifier duty belongs to the typed-decisions checkpoint (0.767 vs Jev 0.727). This export is faithful (torch↔ORT 4.7e-6) but faithfulness is not quality: serve this checkpoint for embedding/compatibility work, not for decisions. CUDA EP drifts 3.2e-3 (kernel order); goldens stay CPU-only.
Serve
from laya.onnx_agent import ONNXAgent
agent = ONNXAgent("convaiinnovations/laya",
onnx_path="laya-base-fp32.onnx", revision="55cf4c4e")
Measured (RTX 3090 + Ryzen 9 5950X)
| Check | Result |
|---|---|
| Graph | opset 18, 1833 nodes, zero non-standard ops |
| torch↔ORT parity | max|Δ| 4.7e-6 on logits (CPU) |
| CPU | ~220 ms/4-row batch — slow-tier only |
| CUDA | ~15 ms/4-row batch — routing-viable |
FP16 variant (laya-base-fp16.onnx + sidecar)
Same graph cast FP32 to FP16 (IO kept FP32), for CUDA serving and halved downloads/VRAM. The rerank lesson holds here too: fp16 is no faster than fp32 on CPU, so this variant is CUDA-only by design.
| Check | Result |
|---|---|
| torch-ORT parity | max abs drift 2.9e-3 on logits (CPU; CUDA drifts comparably) |
| CPU | ~227 ms (no relief, CUDA-only) |
| CUDA | ~11-23 ms (clock-sensitive; controlled rerun pending) |
Dynamic int8 was measured and rejected (drift 2.8-5.4 on the
logit scale with a 3/4 argmax flip on the probe: the masked-logit
head needs calibration-aware static quant, not dynamic). See
manifest.json.
q8
laya-base q8: as td. Gate: worst 0.022, 0 flips, 7/7 golden agree. Parity: worst 0.022, 0 flips (7/7 golden agree).