ViT fine-tuned on Food-101 for Image Classification

Model Description

This model is a fine-tuned version of google/vit-base-patch16-224-in21k for food image classification.

It was fine-tuned on a 5,000-image subset of the Food-101 training split to classify images into the 101 Food-101 categories.

The model was created as a practical exercise while following the Hugging Face LLM Course.

• Developed by: Driw0x
• Model type: Vision Transformer (ViT)
• Base model: google/vit-base-patch16-224-in21k
• Task: Image classification
• Number of labels: 101

Training and Evaluation Data

The model was fine-tuned on:

ethz/food101

The training notebook loads:

food = load_dataset("ethz/food101", split="train[:5000]")
food = food.train_test_split(test_size=0.2)

This produces an 80/20 split of the selected 5,000 examples:

• 4,000 examples for training;
• 1,000 examples for evaluation.

The evaluation split is created from the same 5,000-example subset and is used during model development, so it should not be considered an untouched final benchmark.

Preprocessing

Images are processed with the image processor associated with the base ViT checkpoint.

The training transformation applies a random resized crop, converts the image to a tensor, and normalizes it with the model image mean and standard deviation:

normalize = Normalize(
    mean=image_processor.image_mean,
    std=image_processor.image_std,
)

_transforms = Compose([
    RandomResizedCrop(size),
    ToTensor(),
    normalize,
])

Batch construction uses DefaultDataCollator.

Training Procedure

The model was fine-tuned with the Hugging Face Trainer API.

Training Hyperparameters

Accuracy was computed with the Hugging Face evaluate library.

Training Results

Final reported evaluation results:

• Validation loss: 1.6242
• Accuracy: 0.896

Usage

from PIL import Image
from transformers import pipeline

classifier = pipeline(
    "image-classification",
    model="Driw0x/my_awesome_food_model",
)

image = Image.open("food.jpg")
print(classifier(image))

Direct Inference

import torch
from PIL import Image
from transformers import AutoImageProcessor, AutoModelForImageClassification

model_id = "Driw0x/my_awesome_food_model"

processor = AutoImageProcessor.from_pretrained(model_id)
model = AutoModelForImageClassification.from_pretrained(model_id)

image = Image.open("food.jpg").convert("RGB")
inputs = processor(image, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_id = logits.argmax(-1).item()
print(model.config.id2label[predicted_id])

Intended Uses

This model is primarily intended for:

• learning image classification with Transformers;
• experimenting with ViT fine-tuning;
• reproducing a Hugging Face image-classification workflow;
• educational demonstrations on Food-101.

Limitations

Important limitations include:

• training used only 5,000 images rather than the complete Food-101 training set;
• the evaluation split was derived from the same selected subset;
• the model only predicts the 101 Food-101 categories;
• performance may degrade on unusual viewpoints, low-quality images, mixed dishes, or out-of-distribution food images;
• biases and limitations inherited from the base model and training data may remain.

This model was created as a course exercise and has not been validated for production or high-stakes use.

Framework Versions

• Transformers 5.18.0
• PyTorch 2.11.0+cu130
• Datasets 4.8.5
• Tokenizers 0.23.2

Training Source

The complete training procedure is available in:

Driw0x/hf-ai-courses

Notebook:

llm-course/1-transformer-models/notebooks/image_classification.ipynb