LFM2.5-Encoder-350M-Diffusion-ONNX

ONNX export of LiquidAI/LFM2.5-Encoder-350M-Diffusion, quantized to run fully in the browser through transformers.js. No inference server: the weights are fetched once, cached, and every forward pass happens in the tab.

A diffusion language model. It is not a decoder and emits no token stream: generation starts from a canvas of <|mask|> tokens and denoises it, committing only the most confident predictions each pass and re-guessing the rest with full sight of what landed on either side of them.

All credit for the model itself goes to Liquid AI. This repository contains only a re-export; the weights are unchanged apart from quantization, and the original LFM Open License v1.0 applies.

Try it in your browser → — no install, no API key.

Tooling, demo and the export pipeline: https://github.com/kucukkanat/lfm-encoders

Files

The graph takes input_ids + attention_mask, is dynamic in batch and sequence, and returns logits and last_hidden_state.

Usage

import { AutoTokenizer, PreTrainedModel, Tensor } from "@huggingface/transformers";

const id = "kucukkanat/LFM2.5-Encoder-350M-Diffusion-ONNX";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await PreTrainedModel.from_pretrained(id, { dtype: "q8" });

const { input_ids } = tokenizer("some text");
const out = await model({
  input_ids,
  attention_mask: new Tensor("int64", new BigInt64Array(input_ids.dims[1]).fill(1n), input_ids.dims),
});

PreTrainedModel rather than AutoModel is deliberate: this is a plain "feed the named inputs, read the named outputs" session, not one of transformers.js's built-in architectures.

Generating

The graph is a plain masked-LM forward; what makes it a chatbot is the loop around it, and the loop's schedule ships in config.json under diffusion so consumers do not have to hard-code it:

[Question]
<your question>
[/Question]

[Answer]
<max_new_tokens copies of <|mask|>>

Each pass predicts every still-masked position, and the scheduler commits a subset:

1. Blocks. Unmasking is confined to a block_size window sweeping left to right. Without it the model scatters confident punctuation across the whole canvas and then has to write prose around it.
2. Confidence. Within the block, candidates are ranked by softmax probability. Anything above tau is committed immediately; otherwise just enough are taken to keep the block inside its step budget.
3. Adjacency. Two neighbouring positions are never committed in the same pass — each was predicted while the other was still masked, so both are individually likely and jointly often not.

Ids at or above real_vocab_size are alignment padding and must be excluded from the argmax and the softmax denominator.

@lfm-encoder/tasks implements the whole loop, including frame-by-frame callbacks for animating the denoising.

The reference implementation caches K/V and shortconv state so later passes recompute only the active block. This export has no cache inputs and re-runs the full canvas each pass instead — exact, simpler, and a constant factor more compute.

Accuracy

Measured from JavaScript against the fp32 PyTorch reference. Δ is the largest absolute difference in a final probability.

Per-logit error is the wrong metric for a generative loop — error that never moves an argmax is free, and error that does is compounded by every later pass. So this is measured by decoding the same prompts greedily with each dtype and counting generated tokens that differ from the fp32 PyTorch decode (3 prompts, 32-token canvas, 16 passes).

fp32 is token-identical to PyTorch. q8 paraphrases rather than degrades — it still answers the question — but it is not the same token stream.

q4 is not published: it fails the export's own cosine-similarity gate against fp32 (0.85, threshold 0.90). A one-shot encoder can absorb that; a loop that conditions each pass on the last cannot.

Notes

• q8 is smaller on disk but uses more browser RAM than fp32 and runs slower: onnxruntime's WASM kernels compute in float, so quantized weights are unpacked at session load. Quantization here buys download size, not speed or memory.
• Budget roughly 1.5 GB of RAM per resident model, and expect a tab to hold its high-water mark until reloaded.
• fp16 / q4f16 are deliberately absent: RMSNorm's variance overflows fp16 on this architecture and every hidden state collapses to zeros.