minipfn TabPFN (v1 reproduction)

A from-scratch reproduction of TabPFN (Hollmann et al., arXiv:2207.01848, ICLR 2023), trained with the paper's prior and budget. Code: Aiden-Jeon/minipfn (the tabpfn_v1 modules under src/minipfn/, numbers in src/minipfn/recipes/tabpfn_v1.py).

This is not the official TabPFN checkpoint. For the official models see PriorLabs/TabPFN.

Model

• Architecture: TabPFN v1 Transformer (12 layers, emsize 512, 4 heads, post-norm), 25.82M parameters
• Inputs: up to 100 numerical features, up to 10 classes, up to 1,000 training rows
• Prior: v1 prior bag (SCM and BNN with differentiable hyperparameters, GP) ported to PyTorch
• Training: 18,000 steps, batch 512 datasets, 8×H100 (about 2 hours)

Training

Figures and numbers on this card come from the weights in this repo (sha256:0a13d4b61510, the first 12 hex digits of the SHA-256 of model.safetensors). Train CE is the mean loss over each 25-step window, every step on a freshly sampled prior batch; val CE and accuracy are on 64 fixed prior datasets, logged every 500 steps. Final val CE 0.493, val accuracy 0.770.

Results (paper Table 1)

Mean ROC AUC over the 18 OpenML-CC18 datasets in Table 1 (5 splits each).

Per dataset, against the best of the three sklearn defaults:

Usage

uv pip install "minipfn @ git+https://github.com/Aiden-Jeon/minipfn"

from minipfn import TabPFNv1Classifier

clf = TabPFNv1Classifier().fit(x_train, y_train)  # downloads aiden-jeon/minipfn-tabpfn-v1
proba = clf.predict_proba(x_test)  # (n_test, n_classes), columns in clf.classes_ order

fit does not train weights; it prepares the training rows as context. fit_mode sets how much of that work happens in fit. Both modes return the same probabilities:

• "fit_preprocessors" (default): caches the preprocessing. Use it for batch prediction.
• "fit_with_cache": also caches each layer's training-row keys and values, so predicting one row only computes test-to-train attention. Use it for serving with a fixed training set. The cache takes about 1.6 GB at 1,000 training rows.

If the training rows change with every request, use minipfn.recipes.tabpfn_v1.tabpfn_predict_proba(model, x_train, y_train, x_test). To reproduce Table 1:

uv run python scripts/tabpfn/eval.py --hub-repo aiden-jeon/minipfn-tabpfn-v1

Files

• model.safetensors: weights
• config.json: TabPFNConfig used for training (model size, prior and optimizer settings)
• NOTICE: attribution notices required by Apache-2.0
• images/: figures used in this card

License

Apache-2.0. The prior and evaluation code are derived from the official TabPFN v1 code (automl/TabPFN, branch tabpfn_v1, Apache-2.0, Copyright 2022 The University of Freiburg); see NOTICE.

Citation

@inproceedings{hollmann2023tabpfn,
  title     = {TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second},
  author    = {Hollmann, Noah and M{\"u}ller, Samuel and Eggensperger, Katharina and Hutter, Frank},
  booktitle = {International Conference on Learning Representations},
  year      = {2023}
}