LFM2.5-Encoder-350M-Prompt-Router-ONNX

ONNX export of LiquidAI/LFM2.5-Encoder-350M-Prompt-Router, quantized to run fully in the browser through transformers.js. No inference server: the weights are fetched once, cached, and every forward pass happens in the tab.

A zero-shot prompt router. Categories are ordinary prose supplied at call time — nothing is trained or cached per label set. One bidirectional pass over the category list and the text scores every category at once.

All credit for the model itself goes to Liquid AI. This repository contains only a re-export; the weights are unchanged apart from quantization, and the original LFM Open License v1.0 applies.

Try it in your browser → — no install, no API key.

Tooling, demo and the export pipeline: https://github.com/kucukkanat/lfm-encoders

Files

The graph takes input_ids + attention_mask, is dynamic in batch and sequence, and returns token_proj and rule_proj.

Usage

import { AutoTokenizer, PreTrainedModel, Tensor } from "@huggingface/transformers";

const id = "kucukkanat/LFM2.5-Encoder-350M-Prompt-Router-ONNX";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await PreTrainedModel.from_pretrained(id, { dtype: "q8" });

const { input_ids } = tokenizer("some text");
const out = await model({
  input_ids,
  attention_mask: new Tensor("int64", new BigInt64Array(input_ids.dims[1]).fill(1n), input_ids.dims),
});

PreTrainedModel rather than AutoModel is deliberate: this is a plain "feed the named inputs, read the named outputs" session, not one of transformers.js's built-in architectures.

Prompt format

Both projection towers expect one string laid out exactly like this — the model was trained on it and the character arithmetic that locates each label depends on it byte for byte:

Categories:
- label one
- label two

Text:
<the text>

token_proj is the query tower and rule_proj the key tower, both 256-d and emitted per token. Pool the tokens covering each label to get its vector. Pooling after projecting is exact rather than an approximation: both towers are affine, and an affine map commutes with a mean — which is what keeps one static graph usable for any number of labels.

Scoring is cosine between the L2-normalised pooled towers, scaled by a learned temperature, then a softmax across labels.

@lfm-encoder/tasks implements all of this, including the character-offset reconstruction transformers.js does not provide.

Accuracy

Measured from JavaScript against the fp32 PyTorch reference. Δ is the largest absolute difference in a final probability.

Notes

• q8 is smaller on disk but uses more browser RAM than fp32 and runs slower: onnxruntime's WASM kernels compute in float, so quantized weights are unpacked at session load. Quantization here buys download size, not speed or memory.
• Budget roughly 1.5 GB of RAM per resident model, and expect a tab to hold its high-water mark until reloaded.
• fp16 / q4f16 are deliberately absent: RMSNorm's variance overflows fp16 on this architecture and every hidden state collapses to zeros.