Model Card for mnistvit-transformers

The mnistvit vision transformer (ViT), trained on MNIST with 99.65% test set accuracy, packaged for the Hugging Face transformers library.

The weights are a lossless conversion of the original checkpoint. The model itself is the standard transformers ViTModel with a small classification head, so no dependencies beyond transformers are needed. An ONNX export in onnx/model.onnx runs without Python or PyTorch, for example with ONNX Runtime in C++, C#, Java, mobile apps or the browser.

Model Details

• Developed by: Arno Onken
• Model type: Vision Transformer
• License: MIT
• Original model: asnelt/mnistvit
• Repository: https://github.com/asnelt/mnistvit/
• Demo and MCP server: asnelt/mnistvit Space
• Paper: Dosovitskiy et al., ICLR 2021

How to Get Started with the Model

The model and its image processor are defined in this repository, so loading requires trust_remote_code=True:

from transformers import pipeline

classifier = pipeline(
    "image-classification", model="asnelt/mnistvit-transformers", trust_remote_code=True
)
classifier("sample.jpg")

Or with the model and image processor directly:

import torch
from PIL import Image
from transformers import AutoImageProcessor, AutoModelForImageClassification

repo_id = "asnelt/mnistvit-transformers"
processor = AutoImageProcessor.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForImageClassification.from_pretrained(repo_id, trust_remote_code=True)

inputs = processor(Image.open("sample.jpg"), return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
print(logits.argmax(-1).item())

The image processor reproduces mnistvit.preprocess.read_digit_image: it converts the image to grayscale, center crops and resizes it to 28 by 28 pixels, standardizes it with its own mean and standard deviation, and inverts it if there are more bright than dark pixels. Dark digits on a bright background are therefore classified as well.

To reproduce the original test set evaluation instead, pass normalization="mnist", which normalizes with the fixed MNIST training set mean and standard deviation. This only suits MNIST-like images with bright digits on a dark background:

inputs = processor(Image.open("sample.jpg"), normalization="mnist", return_tensors="pt")

ONNX

onnx/model.onnx (float32, opset 18) includes the per-image standardization and inversion, so it only needs the image as grayscale values:

• Input image: float32, shape (batch_size, 1, 28, 28), grayscale values in [0, 1], center cropped to a square and resized to 28 by 28 pixels
• Output logits: float32, shape (batch_size, 10), one logit per digit

With ONNX Runtime in Python:

import numpy as np
import onnxruntime
from huggingface_hub import hf_hub_download
from PIL import Image

path = hf_hub_download("asnelt/mnistvit-transformers", "onnx/model.onnx")
session = onnxruntime.InferenceSession(path)

image = Image.open("sample.jpg").convert("L")
side = min(image.size)
left, top = round((image.width - side) / 2), round((image.height - side) / 2)
image = image.crop((left, top, left + side, top + side))
image = image.resize((28, 28), Image.Resampling.BILINEAR)
data = np.asarray(image, dtype=np.float32)[np.newaxis, np.newaxis] / 255.0

logits = session.run(None, {"image": data})[0]
print(logits.argmax(-1).item())

The image processor with normalization=None produces exactly this input.

Architecture

The encoder is identical to ViTModel with 28 by 28 single-channel inputs, 4 by 4 patches, 9 layers, 28 attention heads, a hidden size of 420 and an MLP size of 615. Unlike ViTForImageClassification, the classification head is an MLP with one hidden layer of 88 units and GELU activation, as proposed for pre-training by Dosovitskiy et al.

The original implementation also applies dropout after the activation inside each encoder MLP, which ViTModel does not. This only matters when fine-tuning.

Conversion

model.safetensors and config.json were generated with convert_weights.py from asnelt/mnistvit@106c18e. The weights keep their original bfloat16 dtype. onnx/model.onnx was exported from model.safetensors with export_onnx.py. test_parity.py verifies both against the mnistvit package:

• Maximum absolute logit difference on the MNIST test set: 4.8e-6 (float32)
• Identical predictions on all 10,000 test set images
• Image processor matching read_digit_image to within 1e-6
• ONNX model matching the transformers model to within 1e-5, with identical predictions on all test set images

Tested with transformers 4.57.6 and 5.20.0.

Evaluation

Test set accuracy on the 10,000 test set images of MNIST: 99.65%

This figure uses the MNIST training set mean and standard deviation for normalization, as in the original evaluation, and is reproduced by the image processor with normalization="mnist". The default per-image standardization of the image processor and the ONNX model makes the model robust to arbitrary images at the cost of three additional test set errors:

Inverted images have dark digits on a bright background. Low contrast images have pixel values mapped from [0, 1] to [0.3, 0.7].

See the original model card for training details.