Waraqon v3: Arabic OCR with HTML Structure

Fine-tuned Qwen2-VL model for Arabic OCR with HTML formatting, trained on Qari 0.3 dataset.

Model Details

• Base Model: NAMAA-Space/Qari-OCR-0.2.2.1-VL-2B-Instruct
• Training Dataset: QariOCR-v0.3-markdown-mixed-dataset
• Training Steps: 5,000 (checkpoint from interruption)
• Fine-tuning Method: LoRA with Unsloth

Usage

from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
from PIL import Image
import torch

model = Qwen2VLForConditionalGeneration.from_pretrained(
    "FatimahEmadEldin/Waraqon-v3-Arabic-OCR-HTML-Qari",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)
processor = AutoProcessor.from_pretrained("FatimahEmadEldin/Waraqon-v3-Arabic-OCR-HTML-Qari", trust_remote_code=True)

image = Image.open("arabic_text.jpg")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "استخرج النص من هذه الصورة بتنسيق HTML"}
    ]
}]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(text=[text], images=image_inputs, videos=video_inputs, 
                   padding=True, return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(**inputs, max_new_tokens=2048)

generated_ids = [output_ids[len(input_ids):] 
                 for input_ids, output_ids in zip(inputs.input_ids, output_ids)]
output = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(output)

License

Apache 2.0