Stamp, Thumb, and Signature Classifier (ResNet18)

A fine-tuned ResNet18 model for classifying document image regions as stamps, thumbs, or signatures.

Model Details

• Architecture: ResNet18
• Task: Image Classification (3-class: signature, stamp, thumb)
• Input: 224×224 RGB images
• Framework: PyTorch
• Note: This model classifies image regions. Use colored region detection to find candidate regions first.

Performance (Held-Out Validation Set)

• Overall Accuracy: 89.4%
• F1-Score: 88.4%

Performance metrics reported on a held-out validation set (75 documents, 400+ annotations) that was completely separate from training and test data.

Training Dataset Statistics

Original Training Data:

• Signature: 279 images
• Stamp: 100 images
• Thumb: 111 images
• Total Original: 490 images

Augmented Training Data:

• Signature: 1,104 images
• Stamp: 600 images
• Thumb: 783 images
• Total Augmented: 2,487 images

Misclassified Samples (Hard Negatives):

• Signature: 29 images
• Stamp: 27 images
• Thumb: 61 images
• Total Misclassified: 117 images

Augmented Misclassified:

• Signature: 116 images
• Stamp: 108 images
• Thumb: 265 images
• Total Augmented Misclassified: 489 images

Training Set Summary:

• Signature: 1,528 images
• Stamp: 835 images
• Thumb: 1,220 images
• Grand Total: 3,583 training images

Dataset Split

The dataset was split as follows:

Training/Test Split (80-20):

• Training: ~3,000+ images (80% of available data)
• Test: ~20% of available data (used for model development)

Held-Out Validation Set (Performance Evaluation):

• 75 documents
• 400+ annotations
• This held-out validation set was completely separate from training and test data and is used for all reported performance metrics.

Installation

pip install -r requirements.txt

Usage

Download and Load Model

from huggingface_hub import hf_hub_download, HfApi
import torch
import torch.nn as nn
from torchvision import models, transforms
from PIL import Image
import torch.nn.functional as F

repo_id = "Ooredoo-Group/stamp-thumb-signature-classifier-resnet18"

# Access via API first (helps with download tracking)
api = HfApi()
api.model_info(repo_id)

# Download model
model_path = hf_hub_download(
    repo_id=repo_id,
    filename="pytorch_model.bin"
)

# Load model
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
state_dict = torch.load(model_path, map_location=device)
model = models.resnet18(weights=None)
model.fc = nn.Linear(model.fc.in_features, 3)
model.load_state_dict(state_dict)
model.to(device)
model.eval()

class_names = ["signature", "stamp", "thumb"]

Inference

# Preprocess image
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
])
image = Image.open("your_image.jpg").convert("RGB")
input_tensor = transform(image).unsqueeze(0).to(device)

# Predict
with torch.no_grad():
    output = model(input_tensor)
    probabilities = F.softmax(output, dim=1)
    top_prob, top_idx = torch.max(probabilities, dim=1)

result = {
    "label": class_names[top_idx.item()],
    "confidence": top_prob.item(),
    "probabilities": {
        class_names[i]: float(probabilities[0][i].item()) 
        for i in range(len(class_names))
    }
}

print(f"Predicted: {result['label']} (confidence: {result['confidence']:.3f})")

For complete examples including colored region detection, see example_usage.py.

Model Architecture

• Base: ResNet18 (pretrained on ImageNet)
• Final Layer: Linear(512, 3)
• Preprocessing: Resize to 224×224, convert to tensor (no normalization)
• Training: Two-stage progressive transfer learning with data augmentation

Limitations

• Domain: Trained on contract documents (Arabic/English)
• Best Performance: Last 2 pages of documents
• Input Requirements: Requires colored regions (red/blue stamps/thumbs), not black/gray text
• Confidence Threshold: 0.5 (recommended minimum)
• Signature Classification: High recall (99.2%) but moderate precision (84.9%) - may misclassify stamps/thumbs as signatures, especially when signatures overlap faded stamps
• Stamp Classification: Perfect precision (100%) but lower recall (69.5%) - some misses due to rule-based detection method

See Future Improvements for planned enhancements.

Future Improvements

• Handling Overlapping Cases: Expand training dataset to include signatures placed over stamps
• Model-Based Detector: Replace rule-based detection with a model-based detector for better performance
• Enhanced Training Data: Include more diverse samples (faded stamps, obscured regions, various layouts)

Citation

@misc{stamp-thumb-signature-classifier-resnet18,
  title={Stamp, Thumb, and Signature Classifier (ResNet18)},
  author={Ooredoo-Group},
  year={2024},
  howpublished={\url{https://huggingface.co/Ooredoo-Group/stamp-thumb-signature-classifier-resnet18}}
}

License

Apache 2.0