Texteller3 ONNX
Unified, self-contained ONNX serving repo for TexTeller formula OCR
(ViT encoder + TrOCR decoder, VisionEncoderDecoderModel), built from
the union of OleehyO/TexTeller
and Ji-Ha/TexTeller3.
Why one repo
Archaeology (all verified, not assumed):
- Both repos carry identical weights: all 480 torch tensors equal, and the shipped ONNX weight hashes are equal. TexTeller3's ONNX trio is the same checkpoint through a newer optimum export vintage (node counts differ, weights don't).
- Tokenizers are semantically identical (same 11,500 vocab + 3,505 added tokens; only serialization formatting differs).
- Config diffs are version stamps only (
transformers_version,_attn_implementation_autoset). - TexTeller3 adds
preprocessor_config.json(448px grayscale ViT); OleehyO alone shipsdecoder_model_merged.onnx— the file bobine's fp32 path loads.
So this repo takes TexTeller3's files as canonical and adds OleehyO's merged decoder verbatim (weights proven identical by hash, numerics proven identical by gate — see below). No re-export risk anywhere.
Files
| File | Source | Notes |
|---|---|---|
encoder_model.onnx |
TexTeller3 | fp32, opset 18, pixel_values → last_hidden_state [b,785,768] |
encoder_model_dynamic-int8.onnx |
Ji-Ha/TexTeller3-ONNX-dynamic (verbatim) | dynamic int8 encoder, pairs with the int8 merged decoder |
decoder_model_merged.onnx |
OleehyO/TexTeller (verbatim) | fp32, opset 11, 1-node If wrapper (use_cache_branch): no-past + with-past in one file |
decoder_model_merged_dynamic-int8.onnx |
Ji-Ha/TexTeller3-ONNX-dynamic (verbatim, onnx/ subfolder) |
dynamic int8 (weights + activations), pairs with encoder_model_dynamic-int8.onnx |
decoder_model.onnx |
TexTeller3 | fp32, no-past branch standalone |
decoder_with_past_model.onnx |
TexTeller3 | fp32, KV-cache branch standalone |
encoder_model_fp16.onnx |
built here | pure-fp16 compute, fp32 I/O (entry+exit Casts), output names unchanged |
decoder_model_merged_fp16.onnx |
built here | same recipe, If branches included |
model.safetensors |
TexTeller3 (= OleehyO, proven equal) | torch provenance for re-export |
tokenizer.json, vocab.json, merges.txt, … |
TexTeller3 | identical semantics to OleehyO's set |
preprocessor_config.json |
TexTeller3 | 448×448 grayscale, mean 0.9545 / std 0.1539 |
config.json, generation_config.json |
TexTeller3 | bos 0 / eos 2 / pad 1 |
Serving pair (bobine fp32 path): encoder_model.onnx +
decoder_model_merged.onnx + tokenizer.json.
Parity gates (ORT 1.29, CUDA, 3 rendered formulas)
| Check | Result |
|---|---|
| Merged (OleehyO) vs split decoder (T3), branch 0 + branch 1 | worst|d| = 0.0, argmax agree |
T3 shipped ONNX vs fresh optimum export from model.safetensors |
worst|d| = 0.0 |
| fp16 vs fp32, branch 0 / branch 1 | worst|d| ≈ 0.01–0.02, argmax agree |
| fp16 greedy decode strings (32 steps, own cache loop) | exact on 3/3 |
| q8 (MatMulNBits, incl. head-excluded variant) vs fp32 | worst|d| 3.6–7.2, strings 1/3 — SHIPPED ON REQUEST, NOT RECOMMENDED (see below) |
| q8 encoder + fp32 decoder hybrid | diverges — rejected |
The q8 files (encoder_model_q8.onnx, decoder_model_merged_q8.onnx) are published for experimentation only, not for serving: they fail the parity gate (see table) that the house rule requires for serving tiers. The KV-cache loop amplifies int8 noise past the point where greedy strings track, and excluding the LM head does not move the needle. If your decoding pipeline (beam search, repetition penalty, temperature) tracks fp32 output on your data, they cut the download substantially; until then, serve fp32/fp16. Recipe + numbers kept for reproducibility.
Use
import onnxruntime as ort
enc = ort.InferenceSession("encoder_model.onnx", providers=["CUDAExecutionProvider"])
dec = ort.InferenceSession("decoder_model_merged.onnx", providers=["CUDAExecutionProvider"])
hidden = enc.run(None, {"pixel_values": pixels})[0] # [1,1,448,448] normalized per preprocessor_config
logits = dec.run(None, {"input_ids": [[0]], "encoder_hidden_states": hidden,
"past_key_values.0.decoder.key": empty, ..., "use_cache_branch": [False]})
Licence
Apache-2.0 (both upstreams). Formula OCR — image to LaTeX.