useful_charts_table_text_images_vs_useless_images_classifier

This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the codewithaman/useful_charts_table_text_images_vs_useless_images dataset. It achieves the following results on the evaluation set:

• Loss: 0.0851
• Accuracy: 0.9853

To use the model

from transformers import ViTFeatureExtractor, ViTForImageClassification
from PIL import Image
import torch

# Define the device to run the model
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Inference on device: {device}")

# Load the feature extractor and model
model_name_or_path = 'codewithaman/useful_charts_table_text_images_vs_useless_images_classifier'
feature_extractor = ViTFeatureExtractor.from_pretrained(model_name_or_path)
model = ViTForImageClassification.from_pretrained(model_name_or_path).to(device)

# Load local image
def load_image_from_path(image_path):
    image = Image.open(image_path)
    return image.convert("RGB").resize((224, 224))

# Define the inference function
def classify_image(image):
    # Prepare image for the model
    inputs = feature_extractor(images=image, return_tensors="pt").to(device)
    
    # Make prediction
    with torch.no_grad():
        outputs = model(**inputs)
    
    # Extract the predicted label
    predicted_class = outputs.logits.argmax(-1).item()
    label = model.config.id2label[predicted_class]
    return label

# Example usage
image_path = "path/to/your/image.jpg"  # Replace with your local image path
image = load_image_from_path(image_path)
predicted_label = classify_image(image)

print(f"Predicted label: {predicted_label}")

Model description

This model is a Vision Transformer (ViT)-based image classifier fine-tuned on a dataset of images categorized as "useful charts with text" and "useless images." It leverages the google/vit-base-patch16-224-in21k model as its backbone, benefiting from pre-trained weights on a large corpus of general images. This architecture allows the model to capture detailed visual features that distinguish between the two classes effectively, making it particularly useful for identifying informative visual content.

The model takes an image as input and classifies it into one of the specified categories. Its feature extractor processes images into a format compatible with the ViT model, which uses self-attention to understand spatial relationships within images. The model has been optimized for accuracy in distinguishing images based on their content's relevance, focusing on high-level visual features.

Intended uses & limitations

Intended Uses

• Image Classification for Educational Content: Useful for identifying visually rich, informative charts and tables, which can assist in content moderation or educational material curation.
• Content Filtering: Can be used to filter out irrelevant or "useless" images in large datasets where only informational images are desired.
• Dataset Augmentation: Helpful in creating cleaner datasets by selecting images with specific content types, particularly in educational or training datasets.

Limitations

• Generalizability: This model is specifically fine-tuned on images labeled as either useful charts with text or useless images. It may not generalize well to other types of image classification tasks.
• Resolution and Size Constraints: The model's architecture is designed for images resized to 224x224 pixels, so images of significantly different resolutions may affect performance.
• Content-specific Accuracy: Since this model is trained on a specific dataset, it may misclassify images that do not closely resemble the training data (e.g., abstract or artistic images).
• Sensitive Information: This model does not have filters for detecting sensitive or inappropriate content; manual filtering may be required if sensitive content is expected.

Training and evaluation data

The model was trained on the codewithaman/useful_charts_table_text_images_vs_useless_images dataset from the Hugging Face Hub. The dataset contains two main classes:

• Useful Charts and Tables with Text: Images that contain structured, informative visuals like charts, graphs, and tables, often with textual information relevant for educational or informative purposes.
• Useless Images: Images that lack informative content or visual structure useful for educational or analytical purposes.

The training data includes transformations to resize and normalize images, ensuring they are compatible with the ViT model’s input requirements. The evaluation process was carried out on a validation subset, assessing model accuracy and reliability in classifying images into the appropriate categories.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cpu
• Datasets 2.11.0
• Tokenizers 0.20.3