BrailleQwen-2B

BrailleQwen-2B reads a photograph of embossed or printed Braille and writes out the English text it encodes. It is Qwen3.5-2B fully fine-tuned on about 277k synthetic Braille images (Unified English Braille, Grade 1 and Grade 2). No real photograph and no BrailleBench image or label was used for training.

Please read "Evaluation" and "Limitations" before relying on this model. BrailleBench was used as a development set while the training data were designed, so its scores here are probably optimistic.

Model details

How to use

Use transformers 5.x (the checkpoint was saved with 5.10.2; the snippet below has not been run end to end). The training prompt is a single user turn: the image followed by one short instruction, with no system message.

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText

repo = "ai4bharat/BrailleQwen-2B"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")

image = Image.open("braille.png").convert("RGB")
messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": "Transcribe the Braille characters shown in this image."},
    ],
}]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[prompt], images=[[image]], return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
text = processor.tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()
print(text)

• The model was trained with ten paraphrases of the instruction (for example "What Braille text is encoded in this image?", "Read the Braille content in this image and provide the translated text."). Any of them works; the example above is one of the ten.
• Use greedy decoding and set max_new_tokens to a sensible bound for your inputs. On very long or unusual inputs the model can repeat itself; a bound plus a repetition check on your side is advisable.
• Images are resized by the processor; small crops are enlarged automatically. Crops that are tight around the text work best.

Intended use

• Transcribing photographs of English Braille (signs, labels, book and page crops, short passages) into text, for accessibility tools, archiving and research on Braille OCR.
• A baseline for research on synthetic-to-real transfer for Braille recognition.

Out of scope

• Safety-critical or legally binding transcription without human checking (medication labels, emergency signage, contracts).
• Braille in other languages, Nemeth/maths Braille, music Braille, or non-UEB English codes.
• Judging whether an image contains Braille at all, or reading text that is not Braille.

Training

Data

About 277k synthetic images (276,717), one per distinct label text:

1. Text windows of 1-50 words from FineWeb (sample-10BT, plus a smaller CC-MAIN-2013-20 part).
2. Translation to Unicode Braille with Liblouis 3.38.0 (en-ueb-g1.ctb / en-ueb-g2.ctb); 47% of the images are Grade 1 and 53% Grade 2.
3. Rendering as embossed dots with a height-map shading model, then photographic degradations (grain, lighting, perspective, page curl, elastic warp, small rotation, colour casts, low light, noise, JPEG, stains, photo backgrounds).
4. Optional styles: ghost-dot flat print, hollow (ring) dots, coloured tape strips, close-ups; about 25% of the texts have scrambled letters so that the model must read the cells instead of guessing a likely word.

BrailleBench (development set)

Limitations

• Domain. Trained only on our synthetic images; real-photo behaviour is inferred from benchmarks, not from real training data. Different renderings (as the second dataset shows) can degrade accuracy sharply.
• Known weak spots from error analysis of earlier models trained with the same recipe: very large close-ups (above about 70 px per cell), flat-print or ghost-dot styles, dense multi-line pages (repetition loops), low-contrast and very small images, and inserted extra words on very short crops.
• Not covered. Images rotated by 90 degrees, mirrored images, camera-optics distortions, and any Braille other than English UEB.
• Text prior. The model has a language prior from web text. When cells are unreadable it may output a plausible word instead of flagging uncertainty, and it gives no confidence score. Scrambled-letter training reduces but does not remove this.
• Contracted Braille. Grade-2 behaviour follows Liblouis's en-ueb-g2 table; other contraction conventions (for example US EBAE) were not trained.
• Evaluation limits. Reported numbers rely on a benchmark that influenced the training data design (above) and on an earlier, differently sized version of it.

Responsible use

Transcription errors are silent. For anything that matters to a reader's safety, privacy or legal position, keep a human in the loop and treat the output as a draft.

Licenses and acknowledgements

• Base model: Qwen3.5-2B, Apache-2.0.
• Training text: FineWeb (ODC-By 1.0, derived from Common Crawl); no web text is released with this model.
• Braille translation: Liblouis (LGPL-2.1+) used as a tool to create labels; rendering used GNU FreeMono glyphs.