ONNX Export of convaiinnovations/laya

This is an ONNX-converted version of convaiinnovations/laya.

🚀 Quickstart & Usage

This model is a non-autoregressive decision engine exported to ONNX. It evaluates state and questions in a single forward pass using special marker tokens.

Python (onnxruntime)

pip install onnxruntime transformers numpy

import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer

# 1. Load tokenizer and ONNX session
repo_id = "your-username/model-name-onnx"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
session = ort.InferenceSession("laya_int8.onnx")  # or laya.onnx

# 2. Format question & state into Laya's sequence format
# Format: [CLS] <qtype> question: <instructions> [SEP] [MASK] <opt0> [MASK] <opt1> [SEP] <state> [SEP]
state = "The database disk is 100% full and writes are failing."
instructions = "Is this an urgent production blocker?"
options = ["false", "true"]

# Build token IDs & note [MASK] marker positions (id: 50284)
prompt = f"noul question: {instructions}"
head_ids = tokenizer(prompt, add_special_tokens=False)["input_ids"]

opt_ids = []
for opt in options:
    opt_ids.extend([tokenizer.mask_token_id] + tokenizer(f" {opt}", add_special_tokens=False)["input_ids"])

# Assemble sequence
ids = [tokenizer.cls_token_id] + head_ids + [tokenizer.sep_token_id]
markers = []
for i, token in enumerate(opt_ids):
    if token == tokenizer.mask_token_id:
        markers.append(len(ids) + i)
ids.extend(opt_ids)
ids.append(tokenizer.sep_token_id)

state_ids = tokenizer(state, add_special_tokens=False)["input_ids"][:256]
ids = ids + state_ids + [tokenizer.sep_token_id]

# 3. Run Inference (~15ms on CPU)
feeds = {
    "input_ids": np.array([ids], dtype=np.int64),
    "attention_mask": np.ones((1, len(ids)), dtype=np.int64),
    "marker_pos": np.array([markers], dtype=np.int64),
    "marker_mask": np.ones((1, len(markers)), dtype=bool),
    "qtype": np.array([2], dtype=np.int64),  # 0=choice, 1=score, 2=noul
}

logits = session.run(None, feeds)[0][0]

# Softmax probability for P(true)
exp_logits = np.exp(logits - np.max(logits))
probs = exp_logits / np.sum(exp_logits)
print(f"Probability True: {probs[1]:.2%}")

---

TypeScript / Node.js (onnxruntime-node)

npm install onnxruntime-node @xenova/transformers

import * as ort from "onnxruntime-node";
import { AutoTokenizer } from "@xenova/transformers";

const session = await ort.InferenceSession.create("./laya_int8.onnx");
const tokenizer = await AutoTokenizer.from_pretrained("your-username/model-name-onnx");

// Ready to evaluate decisions directly inside JS/TS agent harnesses!

Laya (Base model)

Laya is an open-source, non-autoregressive System 1 decision model: give it a state (text, email, ticket, or JSON document) and typed questions, and it returns typed answers with mathematically calibrated probabilities and confidence scores. It never generates text, eliminating parsing errors and hallucinations.

This is the fine-tuned checkpoint, incorporating dedicated email triage (spam, phishing, department routing), conversation trajectory modeling (TD(lambda = 1.0)), and per-cardinality temperature calibration.

Architecture

• Backbone: ModernBERT-large (395M, fully fine-tuned, bidirectional), plus a decision head trained from scratch (2 transformer layers, an option marker scorer, and an act/escalate head). Total parameters: 421M.
• Option Markers: Every option is scored at its own [MASK] marker token, then a softmax is applied over that question's options.
• Input budget: 512 tokens per question (question + options + state).
• Multi-question batching: Evaluates all questions in a single forward pass (~33 to 38 ms on GPU).

Training

Trained with RLCD (Reinforcement Learning for Calibrated Decisions): the policy reports a probability distribution, exploration adds zero-mean Gaussian noise to the logits, and the reward is a strictly proper scoring rule (log score + spherical score, plus ranked probability score for ordinal score questions). The maximum expected reward is achieved only when the model outputs true, calibrated probabilities.

Multi-turn dialogues use Temporal Difference learning with Monte Carlo targets (TD(lambda = 1.0)) over prefix slices, preventing outcome leakage. 100% human-annotated real-world datasets, zero synthetic shortcuts.

• Training: Fine-tuned (7,313 updates, 1 epoch, ~1.96 hours)
• Fitted calibration temperatures: [1.637, 1.251, 1.983] (with per-option-count scaling)

Benchmark: Laya vs. TypeSafe Jev

Evaluation Results (This Checkpoint)

• In-task test sets: macro accuracy 0.838, macro ECE 0.060
• Zero-shot (held-out task families): macro accuracy 0.651, macro ECE 0.207
• Key Task Families:
  • Intent and routing: accuracy 0.991, ECE 0.009
  • Moderation and safety: accuracy 0.967, ECE 0.061
  • Emotion and tone: accuracy 0.906, ECE 0.018
  • Email triage and phishing: accuracy 0.732, ECE 0.017
  • Inference and fact checking: accuracy 0.883, ECE 0.054

Detailed results by task family, reliability diagrams, and risk-coverage curves are located in eval/ in this repository.

Interactive Web Demo

Try the live Gradio Space: convaiinnovations/laya-demo

License and Support

Released under the Apache 2.0 License by Convai Innovations, who offer commercial support, enterprise integration, and custom fine-tuning.

Limitations

• Text only, English, 512 tokens per question (longer states are truncated).
• Calibration is measured on the benchmark datasets; evaluate on your own distribution before full automation.
• Arithmetic, counting, date comparisons, and multi-hop index lookups should be kept in deterministic code.