RT-J

Relational Transformer checkpoint for in-context prediction over relational databases: one model, binary classification and regression over entities, no per-task training. Pretrained on PluRel, then on the Join. #1 in-context model on the RelBench leaderboard at submission time.

85M parameters, 12 blocks, d_model 512, 8 heads, d_ff 2048; text columns embedded with all-MiniLM-L12-v2. Files: model.safetensors, config.json.

Code, quickstart notebook and recipes: https://github.com/stanford-star/relational-transformer. Paper: RT-J (NeurIPS 2026).

from rt.model import load_rt_model
model = load_rt_model("stanford-star/rt-j")

License

Apache-2.0 for the weights. Training data carries its own terms.

Citation

@inproceedings{ranjan2026rtj,
    title={{RT-J}: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions},
    author={Rishabh Ranjan and Vignesh Kothapalli and Harshvardhan Agarwal and Charilaos Kanatsoulis and Roshan Upendra and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
    booktitle={The Fortieth Annual Conference on Neural Information Processing Systems},
    year={2026}
}

@inproceedings{ranjan2026relational,
    title={Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data},
    author={Rishabh Ranjan and Valter Hudovernik and Mark Znidar and Charilaos Kanatsoulis and Roshan Upendra and Mahmoud Mohammadi and Joe Meyer and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026}
}