dfine-egret-x-rukopys

A D-FINE-X (DFineForObjectDetection, ~62.7M params) document-layout detector for handwritten Ukrainian pages, fine-tuned on the Rukopys dataset with a 7-class head: 0 handwritten · 1 printed · 2 formula · 3 table · 4 annotation · 5 image · 6 graph.

TL;DR

Intended use

Detecting and classifying page regions on handwritten Ukrainian documents, upstream of region recognizers (e.g. Hukyl/trocr-large-rukopys for text, Hukyl/trocr-base-rukopys-formula for formulas). For a stronger single detector on the same task, see Hukyl/doclayout-yolov10-rukopys.

How to use

import torch
from PIL import Image
from transformers import AutoImageProcessor, AutoModelForObjectDetection

repo = "Hukyl/dfine-egret-x-rukopys"
model = AutoModelForObjectDetection.from_pretrained(repo).eval()
processor = AutoImageProcessor.from_pretrained(repo)

image = Image.open("page.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

# threshold ~0.5-0.6 is a balanced operating range
target_sizes = torch.tensor([image.size[::-1]])  # (height, width)
result = processor.post_process_object_detection(
    outputs, threshold=0.5, target_sizes=target_sizes
)[0]
for score, label, box in zip(result["scores"], result["labels"], result["boxes"]):
    print(model.config.id2label[label.item()], round(score.item(), 3), box.tolist())

Files

model.safetensors, config.json,   the shipped checkpoint — loads directly
preprocessor_config.json          with from_pretrained
training_meta.json                recorded recipe + the shipped checkpoint's metrics
metrics.json                      val metrics, aggregate + per-class
selection_comparison.json         the two checkpoint-selection axes side by side
training_log.jsonl                per-epoch loss / mAP / P / R curves

Head re-initialisation (17 → 7)

The backbone, encoder, decoder, and the class-agnostic box-regression head load from the base checkpoint as-is; the classification head is re-initialised to the 7 Rukopys classes, with id2label/label2id baked into the config.

Training

Fine-tuned on the human-labeled Rukopys train split via the 🤗 Trainer. The image processor applies a deterministic 640×640 resize with ImageNet normalisation.

Hyperparameters (as launched)

The full recorded recipe ships in training_meta.json; per-epoch curves in training_log.jsonl.

Checkpoint selection

The run tracked two selection axes; val loss and val mAP@50 are anti-correlated across this run (Pearson ≈ −0.81). This repo ships the mAP@50-selected checkpoint.

Class distribution (region counts)

Results

Held-out val split of the Rukopys train data: 133 pages / 2,629 regions (stratified holdout, seed 42). mAP is threshold-free with box matching at IoU 0.50 (mAP@50–95 averages IoU 0.50:0.95); precision and recall are reported at a fixed confidence ≥ 0.50 operating point (class-aware greedy matching at IoU ≥ 0.50). Per-class precision/recall are not recorded.

Aggregate (macro over 7 classes)

Per class

We also acknowledge that printed/table/annotation/image/graph n is quite small, so measuring detection metrics against them is quite noisy.

Limitations

• annotation (mAP@50 0.26) and image (mAP@50 0.45) are weak classes.
• Trained and evaluated at 640×640; very small or dense regions on high-resolution scans may benefit from a higher inference resolution.
• Precision/recall are reported at the conf ≥ 0.50 operating point; calibrate the inference threshold against your own target metric.
• Handwritten Ukrainian school/archival-style pages only; behaviour on other document types is untested.
• Single seed and validation split — no across-run variance estimate.

Training data & attribution

Model weights are Apache-2.0, inherited from D-FINE and the base checkpoint.