LightOnOCR-3-Poneglyph

Native 0.8B fine-tune for French manga speech bubbles. Input: one RGB bubble crop. Output: its text, with case, accents and punctuation preserved. Use the official empty transcription prompt (image only) and enable_thinking=False.

For full-page detection and bounding boxes, use LightOnOCR-3-Poneglyph-bbox.

Evaluation

Frozen page-disjoint split: 7,496 training crops (860 pages), 1,648 validation crops (184 pages), 1,612 test crops (185 pages). Pixel-level audit found no identical crops across splits. Initialization is the original LightOn model, revision 36d8636925327c671bd8e479c517a004f7566bd9. The bbox fine-tune was not used, because its training pages overlap this benchmark's validation/test pages.

Checkpoint checkpoint-1175 selected only on full validation CER. Native validation CER after preserving visible quotes: 0.4510%. The historical test was also used to evaluate Surya; it did not select this run's checkpoint or hyperparameters. Metrics use the same frozen crops and scoring as that Surya benchmark. Scoring preserves visible quotes, case, accents and punctuation; only surrounding and repeated whitespace is normalized. Surya's stored raw predictions are rescored with this identical corrected protocol. Historical quote-stripping scores are retained separately for transparency. Raw transcription scores are also provided in benchmark_native.json. This is a local-domain evaluation, not a claim about other datasets.

Usage

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "Remidesbois/LightOnOCR-3-Poneglyph"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="cuda", attn_implementation="sdpa"
)
image = Image.open("bubble.png").convert("RGB")
conversation = [{"role": "user", "content": [{"type": "image", "image": image}]}]
inputs = processor.apply_chat_template(
    conversation, tokenize=True, add_generation_prompt=True,
    enable_thinking=False, return_dict=True, return_tensors="pt"
).to(model.device, dtype=torch.bfloat16)
with torch.inference_mode():
    generated = model.generate(**inputs, max_new_tokens=256, do_sample=False)
text = processor.batch_decode(generated[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]
print(text.strip())

Verified with Transformers 5.14.1 / PyTorch 2.8.0 CUDA 12.8 on RTX 5090. Processor uses aspect-preserving Qwen resizing, 65,536–1,048,576 pixels. No thinking, sampling, coordinates or geometry are included in the targets.

Training and integrity

Full language-model fine-tuning, merger and last four vision blocks: 793,333,056 / 852,985,920 parameters trained. FP32 master weights and AdamW states, BF16 autocast, fused AdamW, SDPA, FLA Gated DeltaNet with SM120 causal-conv1d, Liger RMSNorm/SwiGLU and assistant-only weighted CE. Every bubble contributes a normalized assistant/EOS loss, with mild extra weight for short text and onomatopoeia. Conservative photometric augmentation; length grouping and separate lower learning rates for vision/merger.

Export is standard BF16 safetensors. All 473 native tensors equal the BF16 cast of the CER-selected FP32 checkpoint. Final validation/test use a fresh, unpatched Transformers reload. Weights SHA-256: 45c7aa6054eddd1a879715fdb5a3f208dbfedefe06ec270a2f83d7032e61820f. Native weights, processor, validated ONNX assets and aggregate metrics are published. Training crops, references and per-example predictions remain local.

Reproducible comparison

IDs, pages, references, CER, WER and exact match were independently checked. Paired page-cluster bootstrap, 5,000 resamples: LightOn minus Surya CER 95% interval -0.0716 to +0.0784 percentage points. Surya comparison: surya-bubble-ocr-poneglyph frozen benchmark, dated 2026-10-05.

First phase: decoder LR 1.2e-5, six epochs, 60 warmup updates, cosine decay, effective batch 32 (32 crops / update), mild photometric augmentation. Second phase: initialized from first-phase checkpoint 1175, decoder LR 3e-6, three epochs, ten warmup updates, no augmentation and EOS loss weight doubled. Vision and merger LR multipliers are 0.25 and 0.5 throughout. The first-phase checkpoint was retained; consolidation did not improve native validation CER. Selection compared native BF16 validation CER before opening test. Native validation CER of this export: 0.4510%.

Scoring was corrected before opening this run's test: visible quotes are kept, and only whitespace is normalized for both models. Historical quote-stripping test CER: Surya 0.4706%, LightOn 0.4725%. The paired CER interval above includes zero; these data do not establish a clear CER advantage over Surya. Validation curves show the historical scoring; native candidate selection and test comparison use the corrected protocol.

Annotation audit

34 local crops were visually inspected: 18 targeted validation cases, six training repetition/onomatopoeia cases and ten random training crops (seed 42). Two validation annotations were clearly incorrect (repetition count and an onomatopoeia); other stylized cases remain uncertain. All 16 inspected training crops were consistent, which does not certify the full training dataset. Of 169 non-exact validation outputs under historical scoring, 55 differed only in case (67 of 264 character edits). Annotation capitalization conventions are mixed, especially for onomatopoeia. No labels or splits were changed for these results. Review images and detailed private error examples remain local.

The two training loss curves use different EOS weighting. Test was evaluated once for the retained release, after all checkpoint and hyperparameter decisions. These scores measure this local manga dataset; the paired confidence interval describes uncertainty relative to the historical Surya benchmark.

ONNX and browser WebGPU

1600 / 1,612 literal transcripts identical to the native reference. No empty outputs and no token-limit hits. CER change: +0.0058 percentage points; paired page-cluster bootstrap 95% interval [-0.0125, +0.0283] percentage points (5,000 resamples, seed 42). Loaded model on RTX 5090 / Blackwell: median 176 ms/bubble, p95 238 ms; downloads excluded. Other devices and crop/text lengths will have different latencies.

The test is the same frozen set already evaluated for Surya and native LightOnOCR-3. This measures conversion fidelity, not a new independent benchmark. Whitespace is collapsed; visible quotes, capitalization, accents and punctuation are preserved. The native Torchvision uint8 antialiased bicubic resize is reproduced in JavaScript; FP16/FP32 arithmetic can still differ from BF16 inference.

Portable ONNX graphs, manifest and fused WebGPU weight index are in onnx/. Poneglyph offers LightOnOCR 3 · bulles beside Surya and Falcon for bubble crops. Empty image-only prompt, thinking disabled, 256-token budget; visible quotes are kept. Complete browser measurements and asset hashes.