IlhaOCR: One Recogniser for Taiwanese Documents, With a Review Score
Release Summary. IlhaOCR is an optical character recognition (OCR) engine for Taiwanese identity cards, health insurance cards and clinical paperwork. It ships one recogniser. We fine-tuned a PaddleOCR PP-OCRv6 backbone for Taiwanese Traditional Chinese documents. A review branch is attached to it, so a single file both reads a line and reports the probability that the line was read correctly. Everything runs on the central processing unit (CPU) of the machine in front of you, with no network call and no cloud API.
What ships
| File | Role | Bytes | MB |
|---|---|---|---|
PP-OCRv6_rec_tw_fused.onnx |
The deployment artefact: recogniser with the review branch | 23,061,173 | 23.06 |
PP-OCRv6_det_small.onnx |
Text detector | 9,929,594 | 9.93 |
ch_ppocr_mobile_v2.0_cls_mobile.onnx |
Text-direction classifier | 585,532 | 0.59 |
PP-OCRv6_rec_small.onnx |
The same recogniser without the review branch | 22,836,843 | 22.84 |
model_int8.onnx |
Compact build from 2024, kept as a fallback | 35,700,136 | 35.70 |
model.onnx |
Compact build from 2024 in float precision | 44,254,217 | 44.25 |
Sizes are decimal, so 1 MB is 1,000,000 bytes. Every file is an Open Neural Network Exchange (ONNX)
graph and runs on onnxruntime alone. No PyTorch, no CUDA, no GPU.
Also published, for training-time use only: PP-OCRv6_rec_tw_gemma_v2.onnx (22.88 MB),
PP-OCRv6_rec_tw_gemma_fused.onnx (23.06 MB), jev/adapter/ (96.69 MB), jev/head.pt (8,125 bytes)
and jev/jev_config.json. These belong to the line described further down, which is not the
product.
The interface
The deployment artefact takes a three-channel BGR image. The height is fixed to 48 pixels and the
width scales with the crop. The values are normalised to [-1, 1] and the input is named x. The
graph returns two outputs:
| Output | Shape | Meaning |
|---|---|---|
fetch_name_0 |
(1, T, V) | Character probabilities over the 18,710 class vocabulary |
jev_score |
(1,) | Probability that the transcription matches the document |
The first output has the same shape and meaning as the earlier recogniser, so an engine that reads it keeps working without a change. The second output is the review score.
Why we trained our own recogniser
The un-tuned PP-OCRv6 is an open-source model that anyone can download. It is trained on Simplified Chinese, and it falls apart on Taiwanese documents. On the front of an identity card it reads the printed 統一編號 as 玩一編號. It also writes traditional characters in simplified shape.
We retrained it on Taiwanese identity documents and official paperwork, and on synthetic lines laid
out the way Taiwan lays them out. The teacher is a frozen Google SigLIP-2 vision model
(google/siglip2-base-patch16-384, Apache-2.0) read for its upper, middle and lower stroke bands.
The student keeps the PP-OCRv6 backbone and learns those bands through a projection layer. No
language model is involved.
Held-out synthetic card lines
These 374 lines were held out from every training run of every model in this comparison, so they are the cleanest measurement available here.
| Measurement | Un-tuned PP-OCRv6 | Ours |
|---|---|---|
| Exact match rate, spaces ignored | 67.1% | 82.6% |
| Mean character edit distance | 0.86 | 0.64 |
| Exact match rate, spaces sensitive | 36.6% | 34.2% |
| Mean recogniser confidence | 0.9453 | 0.9631 |
The synthetic lines carry deliberate spacing, so the space-sensitive exact match rate is sensitive to spacing rather than to characters. The generic model wins that row. The spaces-ignored row is the closer proxy for character accuracy, and there the fine-tuned model leads by 15.5 points.
Verified real card crops
Thirty hand-verified crops from real cards. Both fine-tuned models saw these during training, so this row is an upper bound and not a clean estimate.
| Measurement | Un-tuned PP-OCRv6 | Ours |
|---|---|---|
| Exact match rate | 73.3% | 100.0% |
| Mean character edit distance | 0.50 | 0.00 |
| Mean recogniser confidence | 0.9609 | 0.9959 |
Side-by-side readings
A separate run compared the un-tuned recogniser with the fine-tuned one over 33 cropped fields and 109 detected boxes from five sample documents. It is not an accuracy or character error rate measurement, because that sample set carries no ground-truth transcription. Only the difference between the two readings is meaningful.
| Measurement | Un-tuned PP-OCRv6 | Fine-tuned (this line) |
|---|---|---|
| Cropped fields with a different reading | — | 10 |
| ... of which simplified characters corrected to traditional | — | 6 |
| Mean recogniser confidence over the cropped fields | 0.9307 | 0.9767 |
| Detected boxes with a different reading | — | 28 |
| ... of which simplified characters corrected to traditional | — | 18 |
| Mean recogniser confidence over the detected boxes | 0.9115 | 0.9310 |
The corrections are the traditional forms the un-tuned model wrote in simplified shape: 性別, 役別,
換發, 保險, 統一編號, and the card header. The same run repairs field structure: a missing 姓 in the
name field, a missing leading zero in the health-insurance card number, and 營 and 鄰 in the address.
A few very low-resolution boxes stay wrong in both models. The aggregation script and its output are
in eval/ (summarise_verification.py, comparison-summary.json).
The review branch
A recogniser returns a string and a confidence score. Neither says whether the string is right. The confidence score is the confidence of each character, not a judgement on the field.
The review branch is 44,417 parameters inside the recogniser. It reads the verified sequence and the connectionist temporal classification (CTC) posterior of the same forward pass and returns the probability that the text just produced is correct. The branch learns from the model's own errors. Every crop the recogniser reads becomes one training item. The label says whether its output matched the known text of that crop.
What it measures
The scorers below judged the same pairs. Each received a crop and the transcription the recogniser produced for it. The label says whether that transcription matched the known text. The evaluation set is 289 held-out synthetic crops, and no crop or text is shared with the training set.
| Measurement | Review branch | Raw confidence |
|---|---|---|
| Area under the receiver operating characteristic curve (AUROC) | 0.968 | 0.911 |
| Wrong readings caught at threshold 0.5 | 86.7% | 8.9% |
| Correct readings kept at threshold 0.5 | 98.8% | 100.0% |
| Wrong readings caught at threshold 0.3 | 84.4% | 6.7% |
| Correct readings kept at threshold 0.3 | 99.6% | 100.0% |
| Extra latency, one thread | 0.6 to 1.0 ms | — |
Two scorers that use only the summary statistics of the forward pass, without the branch, reach an AUROC of 0.907 and 0.907. With the labels permuted, the branch scores fall to 0.478, which is chance. The ranking is not an artefact of the data.
The threshold has to be picked on your own data. The 0.5 above is a measurement point.
On real documents
The branch was trained on Taiwanese identity card crops, and it separates right from wrong readings
on real photographs of cards. On a vaccination record it scored 0.00 for a misread COVID-19(A2)
and 0.08 for NALHEALTHINSURANCE, while the health insurance card number scored 0.9999 and the date
scored 0.9995. A layout it never saw still trips it: a correct PPV scored 0.01.
The earlier reviewer, and why Gemma 4 is not in the product line
The first answer to "is this line right" was a separate reviewer. It was a Gemma 4 model with a low-rank adaptation (LoRA) adapter. It took the crop, a clean rendering of a candidate transcription and the candidate text, then returned whether the two agree. That capability is the reason to reach for Gemma 4 at all: it is a language model, so the scoring head can be swapped to rate arbitrary candidate texts. SigLIP-2 is a vision encoder and has no such head, and rating arbitrary candidates is exactly what review needs.
It worked, and it needed 10.25 GB of base weights at the point of use. In the head-to-head that trained the Gemma line, the reviewer reached an area under the receiver operating characteristic curve of 0.858 and kept 69.2% of the correct readings at threshold 0.5. The branch on that same line reached 0.991 and kept 99.8%. On the 30 verified real card crops the Gemma line read 26, where the shipped line reads 30.
Shipping two models was the wrong shape, and shipping a 10.25 GB reviewer for a worse score would
be worse still. The Gemma 4 assets stay published as training-time material, and the product line
ships one file. The training scripts and the comparison tools are in gemma/.
Latency
Measured on a single-threaded onnxruntime session on one core of a 13th generation Intel Core
i7-13700KF, as the median of 60 runs. The timing images are synthetic lines rendered at the same
pixel sizes as the document crops they stand in for.
| Line | Compact build (2024) | Ours | Ours with the review branch |
|---|---|---|---|
| 250 by 48 pixels, a ten-character line | 28.33 ms | 13.94 ms | 14.57 ms |
| 435 by 52 pixels | 27.99 ms | 21.47 ms | 22.06 ms |
| 680 by 51 pixels | 28.54 ms | 34.74 ms | 35.60 ms |
| 790 by 51 pixels, the longest line | 28.29 ms | 40.59 ms | 41.61 ms |
The compact build takes a fixed 320 by 32 grayscale input, so its time barely moves with the width. This recogniser reads at height 48 with a variable width, so its time grows with the line. On short lines it is faster; on the longest line it is slower.
What the product path reads
The recogniser was run through the same detection, line grouping and field assembly that the hi-care identity service uses. The inputs were tight card crops of a real identity card and a real health insurance card. The name, the identity number with its checksum, the birth date and the issue type came out of the front of the identity card. The father, the mother, the spouse and all three address lines came out of the back. The name, the identity number and the card number came out of the health insurance card.
A tight crop is not a phone photo. The product takes handheld photographs. The back of one real identity card was put through 28 photo conditions in a camera model. The card lies on a table. It is seen at a tilt, turned in the frame, out of focus, shaken by a hand and compressed by JPEG. It is also far away, in dim light and in flat light. Every condition runs the whole product path: detection, line grouping, field assembly, the quality gate and the address rule.
| Photo condition | Result |
|---|---|
| Address delivered and correct | 21 of 28 |
| Address read correctly but refused | 1 of 28 |
| Capture refused and sent back to the operator | 6 of 28 |
| Wrong address delivered into the form | 0 of 28 |
All 7 refusals came from the review score. A refusal leaves the address empty and tells the operator to retake the photograph or type the address.
The recogniser's own character confidence cannot make this call. On the photographs it read wrongly, the confidence stayed as high as 0.9701 while the text it produced was wrong.
Two rules carry the rest. Every crop is turned upright before it is recognised, because a photo of a card is rarely square to the lens. A street that does not read like a street is dropped. Two road names in one line fails the rule. So does a section number without its road. So do characters after the last number and a street without a house number. So does a field label, which shows that two lines were read as one. That rule emptied two of the refused address fields, whose readings were cut short or garbled.
These numbers come from one identity card, one layout, and synthetic camera damage applied to a clean crop. They do not cover other card layouts, other people, or photographs taken in the field. A plausible-looking wrong address can still get through, which is why the operator confirms the fields before registration. A deployment should measure its own photographs before it trusts any of these numbers.
A photo-grade retrain that did not ship
The training damage never went past a Gaussian blur of sigma 1.2. It never included a hand shake either. So a recogniser was retrained on a corpus built from the same camera model. It holds 3974 lines with blur up to sigma 3.2, motion blur, noise up to sigma 6, JPEG from 40 to 92 and a wider brightness range. On 374 held-out degraded lines it reached 87.2% exact match and 0.54 characters of edit distance, against 83.7% and 0.62 for the shipped weights.
It was not published. On the 28 photo conditions it delivered one address more and got one of them wrong. One build of it read the clean crop of the same card with a space inside the house number. A second build, trained on the same corpus with clean lines mixed in, read the clean card correctly and still delivered a wrong address on a shaky photo. The shipped weights stay the ones that never put a wrong address into the form.
The review score and the plain recogniser produced the same text on every line. The branch changes what the model says about its own reading, not what it reads.
Quickstart
1. Installation
pip install onnxruntime numpy pillow huggingface_hub
2. Standalone inference
import numpy as np
import onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(repo_id="weemed/IlhaOCR", filename="PP-OCRv6_rec_tw_fused.onnx")
opts = ort.SessionOptions()
opts.intra_op_num_threads = 1
opts.inter_op_num_threads = 1
session = ort.InferenceSession(model_path, opts, providers=["CPUExecutionProvider"])
# The character list travels inside the model metadata.
characters = session.get_modelmeta().custom_metadata_map["character"].splitlines()
idx2char = {i + 1: ch for i, ch in enumerate(characters)}
def predict_crop(crop: Image.Image) -> tuple[str, float]:
"""Return the transcription of one line and the probability that it is right."""
width = max(16, int(round(crop.width * 48.0 / crop.height)))
resized = np.array(crop.convert("RGB").resize((width, 48), Image.Resampling.BILINEAR))
bgr = resized[:, :, ::-1].astype(np.float32)
normalised = ((bgr / 255.0) - 0.5) / 0.5
tensor = normalised.transpose(2, 0, 1)[np.newaxis, ...]
probabilities, review_score = session.run(None, {"x": tensor})
indices = np.argmax(probabilities[0], axis=-1)
chars, previous = [], 0
for index in indices:
if index != 0 and index != previous:
chars.append(idx2char.get(int(index), ""))
previous = index
return "".join(chars), float(review_score[0])
3. Reading a whole card
The detector and the direction classifier are separate files. A minimal line pipeline looks like this. Run the detector to get boxes. Run the classifier to flip boxes that are upside down. Then run the recogniser on each cropped line and keep the review score of every line.
import json
import numpy as np
import onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download
detector = ort.InferenceSession(
hf_hub_download(repo_id="weemed/IlhaOCR", filename="PP-OCRv6_det_small.onnx"),
providers=["CPUExecutionProvider"],
)
classifier = ort.InferenceSession(
hf_hub_download(repo_id="weemed/IlhaOCR", filename="ch_ppocr_mobile_v2.0_cls_mobile.onnx"),
providers=["CPUExecutionProvider"],
)
recogniser = ort.InferenceSession(
hf_hub_download(repo_id="weemed/IlhaOCR", filename="PP-OCRv6_rec_tw_fused.onnx"),
providers=["CPUExecutionProvider"],
)
def classify(crop: Image.Image) -> Image.Image:
"""Turn a line that is printed upside down the right way up."""
width = max(16, min(256, int(round(crop.width * 48.0 / crop.height))))
resized = np.array(crop.convert("RGB").resize((width, 48), Image.Resampling.BILINEAR))
bgr = resized[:, :, ::-1].astype(np.float32)
normalised = ((bgr / 255.0) - 0.5) / 0.5
tensor = normalised.transpose(2, 0, 1)[np.newaxis, ...]
if classifier.run(None, {"x": tensor})[0][0, 1] > 0.8:
return crop.rotate(180, expand=True)
return crop
4. The legacy compact build
model_int8.onnx and model.onnx are the 2024 build. They take a fixed 320 by 32 grayscale input
under the input name input and return the character probabilities only. They stay in the repository
as a fallback. ilha_ocr.py is the runnable example for that build.
What this repository does not publish
Two scripts need the transcriptions of real identity and health insurance cards: tw_real_gt.py and
legacy_samples.py. The values in the published copies are synthetic. The originals stay on the
machine that holds those documents, and the measurements in this model card come from the originals.
Running these scripts reproduces the method, not those numbers, until the local file is put back.
The per-crop verification log is left out of the repository for the same reason. The aggregated summary next to it is what this model card cites.
gemma/ holds the scripts and the comparison tools of the line we tried and did not ship.
Licence
Apache-2.0. See LICENSE.
繁體中文說明
IlhaOCR 是專為台灣身分證、健保卡與臨床文件所研發的光學文字辨識(OCR)引擎。 它只出貨一組辨識器:我們把 PaddleOCR PP-OCRv6 的骨幹微調成台灣繁體中文專用,再把一個覆核分支 接在同一個檔案裡。於是一個模型既讀字,也說明那一行讀對的機率。整條推論路徑在眼前那台機器的 中央處理單元(CPU)上跑完,不連網,也沒有雲端 API。
模型庫裡有什麼
| 檔案 | 角色 | 位元組 | MB |
|---|---|---|---|
PP-OCRv6_rec_tw_fused.onnx |
部署用這個:含覆核分支的辨識器 | 23,061,173 | 23.06 |
PP-OCRv6_det_small.onnx |
文字偵測器 | 9,929,594 | 9.93 |
ch_ppocr_mobile_v2.0_cls_mobile.onnx |
文字方向分類器 | 585,532 | 0.59 |
PP-OCRv6_rec_small.onnx |
同一組辨識器,未接覆核分支 | 22,836,843 | 22.84 |
model_int8.onnx |
2024 年的精簡版,留作備援 | 35,700,136 | 35.70 |
model.onnx |
2024 年的精簡版,浮點精度 | 44,254,217 | 44.25 |
尺寸以十進位計,1 MB 就是 1,000,000 bytes。每個檔案都是開放神經網路交換格式(ONNX),只用
onnxruntime 就能跑,不需要 PyTorch、不需要 CUDA、不需要 GPU。
另外也公開、僅供訓練期使用的檔案:PP-OCRv6_rec_tw_gemma_v2.onnx(22.88 MB)、
PP-OCRv6_rec_tw_gemma_fused.onnx(23.06 MB)、jev/adapter/(96.69 MB)、jev/head.pt(8,125 bytes)
與 jev/jev_config.json。這些屬於下面才會說明的那條線,不是產品。
介面
部署檔吃三通道 BGR 影像,高度固定 48 像素、寬度依比例,正規化到 [-1, 1],輸入名稱為 x。
它回傳兩個輸出:
| 輸出 | 形狀 | 意義 |
|---|---|---|
fetch_name_0 |
(1, T, V) | 18,710 個類別的字元機率 |
jev_score |
(1,) | 這段轉錄正確的機率 |
第一個輸出與前一條線的形狀和意義相同,所以讀它的引擎不用改。第二個輸出就是覆核分數。
為什麼自己訓練辨識器
未微調的 PP-OCRv6 是開源模型,誰都能下載。它是用簡體中文訓練的,在台灣文件上會整個垮掉。 身分證正面的「統一編號」,它會讀成「玩一編號」,繁體字也常用簡體形狀寫出來。
我們用台灣的證件與官方文件重訓它,另外加上一批照著台灣排版合成的文字行。老師是一個凍結的
Google SigLIP-2 視覺模型(google/siglip2-base-patch16-384,Apache-2.0),只讀它的上、中、下
三段筆畫特徵。學生端保留 PP-OCRv6 的骨幹,用一個投影層學習這些特徵。這條線沒有任何語言模型。
留出的合成卡片文字行
這 374 筆任何一條訓練線都沒看過,是這份比較裡最乾淨的一次量測。
| 量測 | 未微調 PP-OCRv6 | 我們的版本 |
|---|---|---|
| 完全相符率(忽略空白) | 67.1% | 82.6% |
| 平均字元編輯距離 | 0.86 | 0.64 |
| 完全相符率(空白敏感) | 36.6% | 34.2% |
| 平均辨識信心 | 0.9453 | 0.9631 |
合成行刻意排版空白,所以對空白敏感的那一列量的是空白,不是字。通用版贏那一列;忽略空白的 那一列比較接近字準,微調版在那裡領先 15.5 個百分點。
人工核過的真實卡片裁切
30 筆來自真實卡片、人工核過的裁切。兩個微調模型在訓練時都看過,所以這一列是上限,不是乾淨估計。
| 量測 | 未微調 PP-OCRv6 | 我們的版本 |
|---|---|---|
| 完全相符率 | 73.3% | 100.0% |
| 平均字元編輯距離 | 0.50 | 0.00 |
| 平均辨識信心 | 0.9609 | 0.9959 |
兩組模型並排跑
另一輪把未微調的辨識器與微調後的辨識器並排跑過 33 個裁切欄位與五份樣本文件的 109 個偵測框。 這不是準確率或字元編輯錯誤率量測,因為那批樣本沒有標準答案,只有「兩個讀法不一樣」這件事有意義。
| 量測 | 未微調 PP-OCRv6 | 微調後(本組) |
|---|---|---|
| 讀法不同的裁切欄位數 | — | 10 |
| ... 其中簡體字修正為繁體字 | — | 6 |
| 裁切欄位的平均辨識信心 | 0.9307 | 0.9767 |
| 讀法不同的偵測框數 | — | 28 |
| ... 其中簡體字修正為繁體字 | — | 18 |
| 偵測框的平均辨識信心 | 0.9115 | 0.9310 |
被修正的是未微調模型以簡體寫出的繁體字:性別、役別、換發、保險、統一編號,以及證件標題。
同一輪比對也修正了欄位結構:姓名欄少了的「姓」、健保卡卡號少的前導零,以及地址裡的「營」與「鄰」。
少數解析度很低的框,兩個模型都還是不對。彙總腳本與輸出放在 eval/(summarise_verification.py、
comparison-summary.json)。
覆核分支
辨識器會回傳字串與信心值,但兩者都沒有說明字串是否正確。信心值是每個字的信心,不是對整欄的判斷。
覆核分支是辨識器裡的 44,417 個參數。它讀同一趟前向運算的驗證序列與連接時序分類(CTC)後驗, 回傳「剛剛產出的那段文字是對的」機率。這個分支從模型自己的錯誤學習:辨識器讀過的每一張裁切都 成一筆訓練資料,標籤是它的輸出有沒有等於該裁切已知的文字。
它量到什麼
下面的評分器判斷同一批配對。每個配對是一張裁切,加上辨識器對它產生的轉錄;標籤是這段轉錄有沒有 等於已知文字。評估集是 289 張留出的合成裁切,裁切與文字都不與訓練集重疊。
| 量測 | 覆核分支 | 原始信心值 |
|---|---|---|
| 接收者操作特徵曲線下面積(AUROC) | 0.968 | 0.911 |
| 閾值 0.5 抓到錯讀 | 86.7% | 8.9% |
| 閾值 0.5 保留正確轉錄 | 98.8% | 100.0% |
| 閾值 0.3 抓到錯讀 | 84.4% | 6.7% |
| 閾值 0.3 保留正確轉錄 | 99.6% | 100.0% |
| 單執行緒額外延遲 | 0.6 到 1.0 毫秒 | — |
另外兩個只用前向統計量、不含這個分支的評分器,AUROC 各是 0.907 與 0.907。把標籤打亂,同一組 分數掉到 0.478,也就是隨機水準。這個排序能力不是資料的巧合。
閾值要在自己的資料上挑。上表的 0.5 只是量測點。
在真實文件上
這個分支是照台灣身分證裁切訓練的,在真實卡片照片上,它分得出讀對與讀錯。疫苗紀錄上誤讀的
COVID-19(A2) 得 0.00,NALHEALTHINSURANCE 得 0.08,健保卡卡號得 0.9999,日期得 0.9995。
沒見過的版面還是會失手:正確的 PPV 只得 0.01。
最早的審查者,以及 Gemma 4 為什麼不在產品線裡
「這行對不對」的第一個答案是另一個獨立的審查者:一個帶低秩適配(LoRA)adapter 的 Gemma 4 模型, 它吃裁切、候選轉錄的乾淨渲染圖與候選文字,然後回傳兩者是否一致。會想到 Gemma 4 就是這個原因: 它是語言模型,所以打分用的頭可以換掉,去評分任意候選文字;SigLIP-2 是視覺編碼器,沒有這種頭, 而評分任意候選文字正是覆核需要的。
它有效,但它在使用的時候要背 10.25 GB 的權重。在訓練 Gemma 那條線的對決裡,它的 AUROC 是 0.858, 閾值 0.5 只保留 69.2% 的正確轉錄;同一條線上的分支是 0.991,保留 99.8%。30 筆人工核過的真實 卡片裁切,Gemma 那條線對 26 筆,出貨的這條線對 30 筆。
出貨兩個模型本來就是錯的形狀,而為了更差的分數去背 10.25 GB 的審查者更糟。Gemma 4 的資產仍然公開,
定位是訓練期材料,產品線只出貨一個檔案。訓練腳本與對照工具在 gemma/。
延遲
單一執行緒的 onnxruntime session,在第 13 代 Intel Core i7-13700KF 的一個核心上量,取 60 次中位數。
計時用的影像是以合成文字排版成與文件裁切相同的像素尺寸。
| 文字行 | 精簡版(2024) | 我們的版本 | 含覆核分支 |
|---|---|---|---|
| 250 乘 48 像素,十個字的行 | 28.33 毫秒 | 13.94 毫秒 | 14.57 毫秒 |
| 435 乘 52 像素 | 27.99 毫秒 | 21.47 毫秒 | 22.06 毫秒 |
| 680 乘 51 像素 | 28.54 毫秒 | 34.74 毫秒 | 35.60 毫秒 |
| 790 乘 51 像素,最長的一行 | 28.29 毫秒 | 40.59 毫秒 | 41.61 毫秒 |
精簡版吃的是固定的 320 乘 32 灰階輸入,所以時間幾乎不隨寬度變動。這個辨識器以 48 像素高度、 寬度可變的方式讀字,時間會隨行長增加。短行它比較快,最長的一行它比較慢。
產品路徑讀到什麼
辨識器跑過 hi-care 身分證服務同一套偵測、行分組與欄位組裝。輸入是身分證與健保卡的裁切乾淨圖。 身分證正面的姓名、身分證號與校驗碼、出生日期與發證類別都讀對了。背面的父親、母親、配偶與三行 地址都讀對了。健保卡的姓名、身分證號與卡號也讀對了。
裁切乾淨圖不等於手機拍的照片。 產品吃的是手持照片,所以拿同一張身分證背面走過相機模型裡的 28 種照片條件:卡放在桌上的斜視、畫面中的旋轉、失焦、手震、JPEG 壓縮、距離、弱光與平光。 每一個條件都跑整條產品路徑:偵測、行分組、欄位組裝、品質閘門與地址規則。
| 照片條件 | 結果 |
|---|---|
| 地址讀對並填進表單 | 21 / 28 |
| 地址讀對但被擋下 | 1 / 28 |
| 擋下、退回給操作人員 | 6 / 28 |
| 讀錯地址又填進表單 | 0 / 28 |
7 個拒收全部來自覆核分數。拒收時地址欄留空,操作人員會被要求重拍或手動填寫。
辨識器自己的信心值分不出對錯:讀錯的案例裡,信心值最高還有 0.9701,而它讀出來的那段文字是錯的。
剩下的部分靠兩條規則。第一,每個框在辨識前先轉正,因為拍證件的照片很少跟證件平行。第二, 不像街道的地址不給:一行兩個路名、沒有路名的段號、最後一個數字之後還有字、沒有門牌號, 以及混入欄位標籤,都算讀壞。這條規則清空了 2 個被拒收的地址欄,那兩個讀到的字是被截斷或亂掉的。
這些數字來自一張身分證、一種版面,而且是在乾淨裁切圖上做出來的相機劣化。它們沒有涵蓋其他 版面、其他人,或現場拍的照片。看起來合理但內容錯的地址仍然可能通過,所以登記前一定要讓操作 人員確認。每一個上線的環境都該先用自己的照片量一次,再決定要不要信這組數字。
一個沒有發布的照片級重訓
原本的訓練劣化最高只有 1.2 的高斯模糊,也從來沒有手震模糊。所以又用同一個相機模型重建了語料: 3974 筆,模糊最高到 3.2σ、手震模糊、雜訊最高 6σ、JPEG 品質 40 到 92,亮度範圍也拉大。 在 374 筆留出的劣化文字行上,完全相符率到 87.2%、平均字元編輯距離 0.54,比現行權重的 83.7% 與 0.62 好。
這個版本沒有發布。 在 28 個照片條件上,它多填對一個地址,也多填錯一個;其中一個版本還把同一張 卡乾淨裁切圖讀成門牌號中間多一個空格。另一個版本混了乾淨文字行,乾淨卡片讀對了,卻還是在一張 手震照片上填了錯的地址。現行權力留的是那個從來沒有把錯地址填進表單的版本。
在每一行上,覆核分支與不帶分支的辨識器讀出的文字完全相同。這個分支改變的是模型怎麼說自己讀得 對不對,不是它讀到什麼。
快速上手
1. 安裝
pip install onnxruntime numpy pillow huggingface_hub
2. 單行推論
import numpy as np
import onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(repo_id="weemed/IlhaOCR", filename="PP-OCRv6_rec_tw_fused.onnx")
opts = ort.SessionOptions()
opts.intra_op_num_threads = 1
opts.inter_op_num_threads = 1
session = ort.InferenceSession(model_path, opts, providers=["CPUExecutionProvider"])
# 字元表就放在模型的中繼資料裡。
characters = session.get_modelmeta().custom_metadata_map["character"].splitlines()
idx2char = {i + 1: ch for i, ch in enumerate(characters)}
def predict_crop(crop: Image.Image) -> tuple[str, float]:
"""回傳一行的轉錄,以及那行讀對的機率。"""
width = max(16, int(round(crop.width * 48.0 / crop.height)))
resized = np.array(crop.convert("RGB").resize((width, 48), Image.Resampling.BILINEAR))
bgr = resized[:, :, ::-1].astype(np.float32)
normalised = ((bgr / 255.0) - 0.5) / 0.5
tensor = normalised.transpose(2, 0, 1)[np.newaxis, ...]
probabilities, review_score = session.run(None, {"x": tensor})
indices = np.argmax(probabilities[0], axis=-1)
chars, previous = [], 0
for index in indices:
if index != 0 and index != previous:
chars.append(idx2char.get(int(index), ""))
previous = index
return "".join(chars), float(review_score[0])
3. 讀整張卡
偵測器與方向分類器是另外兩個檔案。最小的行流程是:先跑偵測器拿到方框,再跑分類器把上下顛倒的行 翻正,最後對每一個裁切跑辨識器,並且保留每一行的覆核分數。
import numpy as np
import onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download
detector = ort.InferenceSession(
hf_hub_download(repo_id="weemed/IlhaOCR", filename="PP-OCRv6_det_small.onnx"),
providers=["CPUExecutionProvider"],
)
classifier = ort.InferenceSession(
hf_hub_download(repo_id="weemed/IlhaOCR", filename="ch_ppocr_mobile_v2.0_cls_mobile.onnx"),
providers=["CPUExecutionProvider"],
)
recogniser = ort.InferenceSession(
hf_hub_download(repo_id="weemed/IlhaOCR", filename="PP-OCRv6_rec_tw_fused.onnx"),
providers=["CPUExecutionProvider"],
)
def classify(crop: Image.Image) -> Image.Image:
"""把印反的行翻正。"""
width = max(16, min(256, int(round(crop.width * 48.0 / crop.height))))
resized = np.array(crop.convert("RGB").resize((width, 48), Image.Resampling.BILINEAR))
bgr = resized[:, :, ::-1].astype(np.float32)
normalised = ((bgr / 255.0) - 0.5) / 0.5
tensor = normalised.transpose(2, 0, 1)[np.newaxis, ...]
if classifier.run(None, {"x": tensor})[0][0, 1] > 0.8:
return crop.rotate(180, expand=True)
return crop
4. 舊的精簡版
model_int8.onnx 與 model.onnx 是 2024 年的版本。它們吃固定的 320 乘 32 灰階輸入,輸入名稱是
input,只回傳字元機率。它們留在模型庫當備援。ilha_ocr.py 是那一版的可執行範例。
這個模型庫不公開什麼
有兩支腳本需要真實身分證與健保卡的轉錄內容:tw_real_gt.py 與 legacy_samples.py。公開版裡的
數值是合成等價物。原值留在持有那些文件的機器上,本模型卡引用的量測來自原值。在把本機檔案放回去
之前,跑這些腳本重現的是做法,不是那些數字。
逐張裁切的驗證日誌也因為同樣的理由不放進模型庫。旁邊的彙總摘要才是本模型卡引用的東西。
gemma/ 放的是那條我們試過、但沒有出貨的線的腳本與對照工具。
授權
Apache-2.0。見 LICENSE。