CaptchaOCR PP-OCRv6 — captcha text recogniser

A PP-OCRv6 fine-tune that reads a captcha image, usually the answer band: 18 evaluated styles, Latin letters, digits, and part of the Cyrillic alphabet.

Use it in 10 lines

import torch
from PIL import Image
from transformers import PPOCRV6SmallRecForTextRecognition
from transformers import PPOCRV6SmallRecImageProcessor

REPO = "Titlehhhh/captchaocr-ppocrv6"
model = PPOCRV6SmallRecForTextRecognition.from_pretrained(REPO).eval()
proc = PPOCRV6SmallRecImageProcessor.from_pretrained(REPO)
inputs = proc(images=[Image.open("band.png").convert("RGB")], return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs)
print(proc.post_process_text_recognition(logits))  # [{'text': 'ly8r', 'score': 0.96}]

band.png is a tight crop of the answer band, not a screenshot. The processor resizes it to 48 × 320, normalises it, and decodes CTC with the 18 762-entry dictionary from preprocessor_config.json. This exact snippet was run against these weights.

Manual CTC decoding

If you would rather decode by hand, the forward returns the logits in last_hidden_state:

with open("character_list.txt", encoding="utf-8") as f:
    dictionary = [line.rstrip("\r\n") for line in f]   # rstrip, not strip: " " is a class
ids = logits.last_hidden_state.argmax(-1)[0].tolist()
text, prev = [], -1
for i in ids:
    if i != prev and i != 0:        # collapse repeats, drop the blank
        text.append(dictionary[i])
    prev = i
print("".join(text))

What it supports

Results

Exact match and CER after greedy CTC decoding, on 11 400 images of 18 styles: 200 rows per trained style plus 1 500 rows per held-out plugin. The median is taken over the 18 per-style scores; the micro average counts every image, and 9 000 of the 11 400 belong to held-out plugins.

Plugins never seen in training — what to expect on a new style:

Per-style results on the trained styles

Sample-level hold-out: the first 200 rows of each style were taken out of training.

Limitations

• Four of the twelve trained styles are fed the whole frame, not a band. Three of them are the weakest: XProtect/code 0.36, XProtect/map 0.27, textcaptcha 0.24. The fourth, AnBot/captcha, scores 0.9950 on the frame — the input is not what makes a style hard.
• The median is carried by the easy styles. Seven of the twelve trained styles score 0.89 or better. On a style unlike the training set, read the held-out table instead: 0.57–0.98.
• Screenshots need a cropper. Feed a raw game frame and 2 of 6 real screenshots are read correctly; with the cropper, 6 of 6. The cropper ships with the code, not with these weights.
• The micro average is mostly held-out data (9 000 of 11 400 images), so it answers "how does it do on unseen plugins", not "accuracy in general".
• It is not an API and not a solver for new plugins. One image in, one string out; a style absent from training has no guarantee.

Dataset

Titlehhhh/captchaocr-dataset is synthetic: every image is a re-render by this project's Java generators of a captcha drawn by an open-source Minecraft plugin. The published snapshot holds 20 styles and 200 000 samples. A 21st style, out_ru/LimboFilter/digits, was generated after the export and is not in the snapshot — which is why 18 evaluated styles = 11 from the snapshot + 1 unpublished + 6 held out.

Training

• Base: PP-OCRv6_medium_rec (Apache-2.0). The CTC head was kept and grown from 18 710 to 18 762 classes: rows 18 710–18 761 are 52 appended Cyrillic characters.
• Initialised from an earlier Cyrillic run of the same recipe that replayed Latin data, then fine-tuned on 12 styles.
• 5 epochs × 488 steps = 2 440 steps, batch 32, learning rate 8e-6 for backbone and head, weight decay 0.01, gradient clipping 1.0.
• 15 594 training examples: 1 300 per style after a 200-row hold-out, from a 10 000-per-style render (the run used --limit-per-type 1500).
• 33.9 min on one GTX 1080 Ti, 108 images/s, fp16.
• Whole-frame styles were trained with the frame as input; the rest with the roi band.
• Full command line and scripts: github.com/Titlehhhh/CaptchaOCR.

License & citation

The weights are Apache-2.0, as a derivative of PP-OCRv6_medium_rec. The training data is published under mixed per-plugin licences — some rows are non-commercial, so read the dataset LICENSE before commercial use.

Version history

• universal_v1 (2026-10-27) — current weights, 18 762 classes, no weight interpolation (α = 1.0). It replaced all20, which is no longer downloadable.
• Interpolation is off on purpose: pulling back towards the base damages the appended rows (LimboFilter/symbols 0.765 → 0.016). Replay of old data, not a weight soup, is what keeps them.