ATF → English (T5-large B2c), ONNX for Transformers.js

ONNX export of montycrypto/atf-to-english (checkpoint B2c / checkpoint-22000, T5-large) for in-browser use with Transformers.js v3. Intended for the Unread Atlas in-browser translator.

The previous T5-small browser build (atf-en-best-v4e) remains available as tag v4e-2026-10-11 (commit 2889e854).

Export: optimum-cli export onnx --task text2text-generation-with-past, then onnxruntime dynamic int8 quantization with per_channel=True and EnableSubgraph for the merged decoder. fp32 ONNX (~3.2 GB) is omitted from this Hub copy; recover it from the source PyTorch weights or the internal Alpaca backup.

Quality (300-line clean mixed-test sample, A4 input, greedy, number-normalised).

CPU latency on the export host (4 threads, batch 1): int8 median 0.16 s / p90 0.60 s per line.

Tokenizer note. tokenizer.json here differs from the PyTorch repo: the Sequence[WhitespaceSplit, Metaspace] pre-tokenizer is replaced by a single classic T5 Metaspace (add_prefix_space: true), because Transformers.js 3.7 mis-tokenizes the original. Checked against the PyTorch tokenizer: identical ids on the 300-line A4+plain sample and on ≈368k training strings; the only differences found were 29 English strings with double spaces or a BOM. For Python, keep using the original tokenizer next to the PyTorch weights.

Input format. B2c was trained with the tokfix pipeline: Unicode ATF → CDLI-ASCII (š→sz, ṣ→s,, ṭ→t,, ḫ→h, accented vowels → index digits, …), then &-escapes for any remaining non-safe characters, then the prefix translate Akkadian to English: . Decode model output with the inverse of those escapes. The Unread Atlas worker applies this when pointed at this revision; raw ATF without that conversion will under-perform.

import { pipeline } from "@huggingface/transformers";
const t = await pipeline("text2text-generation", "montycrypto/atf-to-english-onnx", { dtype: "q8" });
const out = await t("translate Akkadian to English: a-na be-li2-ia qi2-bi2-ma", {
  num_beams: 4, max_new_tokens: 256, repetition_penalty: 1.2,
});

Decoding note. Transformers.js 3.x has no beam search: num_beams is ignored and decoding is greedy. Verified token-for-token against PyTorch greedy decoding of the source model (ONNX fp32). On the 300-line sample, greedy scored higher than the production 4-beam settings (chrF 55.19 vs 53.26), so the browser's greedy path is not a quality regression vs beams here.

Source model: montycrypto/atf-to-english (B2c). Prompt prefix translate Akkadian to English: , one line per call. Output is a machine draft, not a scholarly translation.