Event-level graph autoencoder for the LHC Olympics 2020 R&D dataset (negative result)

This repository contains an unsupervised graph autoencoder. The model was trained on QCD background events from the LHC Olympics 2020 R&D dataset.

The repository gives a reference for a known failure. For the W′ → XY signal of the dataset, the anomaly scores of the model are inverted: their ROC AUC is below 0.5. A simple variable without labels, the mean pT per particle, gets an AUC of 0.81.

• Paper: J. Khan, When Graph Autoencoders Look the Wrong Way: Complexity Bias and Simple Baselines in Unsupervised Anomaly Detection on the LHC Olympics 2020 Dataset (2026), doi:10.5281/zenodo.23432779.
• Code: github.com/Viverun/Hidden-Vertex.
• Results notebook: kaggle.com/code/viveriun/hidden-vertex-gae-results.

Do not use this model as an anomaly detector. Use it for these tasks:

• Reproduce the result of the paper.
• Use it as a negative control for a new method.
• Examine why reconstruction scores follow the event complexity.

Contents

Results (frozen test set: 5,000 background events and 5,000 signal events)

Simple variables on the same events, without labels and without training:

An AUC of 0.40 to 0.44 means that the autoencoder gives background events higher scores than signal events. In this dataset, the signal events have fewer particles, and their particles have higher pT. Thus, the model reconstructs them more easily. For the details, see Section 4 of the paper.

These checkpoints come from a new training run with the configuration and the seeds of the paper. The AUC of each seed is the same as in the paper. The published weights reproduce the stored Chamfer scores of the test events with a maximum difference of 1e-5 (see metrics.json).

Usage

from huggingface_hub import snapshot_download
import sys
path = snapshot_download("Viverun/lhco-rnd-event-gae")
sys.path.insert(0, path)
from hv_event_ae import load, score

model, ctx = load(f"{path}/seed0")
# particles: float32 [N, P, 3] = (pT [GeV], eta, phi) of each particle,
# sorted by pT in decreasing sequence, with zeros in empty slots.
# P must be 48 or more. The paper used P = 350.
# A row of the R&D HDF5 file (columns 0 to 2099) gives an array of shape [700, 3].
chamfer, z = score(model, ctx, particles)   # a higher Chamfer value means "more anomalous"

The code needs torch, numpy and safetensors.

To calculate the kNN and GMM scores, fit the scorers on latents_fit.npy:

1. Standardize each latent dimension with the mean and the standard deviation of latents_fit.npy.
2. kNN: the mean distance to the 20 nearest training latents (scikit-learn NearestNeighbors).
3. GMM: the negative log-likelihood under GaussianMixture(8, covariance_type='full', reg_covar=1e-4, max_iter=200, random_state=seed).

The code repository has this procedure in src/utils/latent_scores.py.

Test-set rows

The column h5_row gives the row in the file events_anomalydetection_v2.h5. The paper used this file. Its rows are sorted: rows 0 to 999,999 are background, and rows 1,000,000 and higher are signal. The test background events are rows 40,000 to 44,999. The test signal events are rows 1,000,000 to 1,004,999.

Training

• Data: 180,000 background events (rows 45,000 and higher) for training and 20,000 background events for validation. The training set does not contain signal events.
• Model: 3 neighbour-averaging layers (width 64, ΔR ≤ 0.4, residual connection and LayerNorm). Then mean, max and pT-weighted pooling, a latent space of 16 values, and an MLP decoder for the 48 particles with the highest pT. The model has 95,888 parameters.
• Loss: symmetric Chamfer distance on (log pT, η, cos φ, sin φ).
• Optimizer: AdamW, learning rate 0.001, weight decay 0.00001, batch size 256, gradient clip 5.0.
• Epochs: 30 (maximum). The training stops after 5 epochs without a better validation loss. This stop did not occur.
• Hardware: one NVIDIA T4 GPU on Kaggle, approximately 18 minutes for each seed.

Limits

• The test uses one signal model: W′ → XY → (qq̄)(qq̄), with m(W′) = 3.5 TeV, m(X) = 500 GeV and m(Y) = 100 GeV.
• The test uses one detector simulation (Delphes).
• The inversion is a property of reconstruction scores for this signal. It does not prove that all graph autoencoders fail. For a signal that is more complex than QCD background, the result can be different.

Citation

@misc{khan2026gae,
  author = {Khan, Jamil},
  title  = {When Graph Autoencoders Look the Wrong Way: Complexity Bias and Simple Baselines in
            Unsupervised Anomaly Detection on the LHC Olympics 2020 Dataset},
  year   = {2026},
  doi    = {10.5281/zenodo.23432779}
}

Also cite the dataset: G. Kasieczka, B. Nachman, D. Shih, R&D Dataset for LHC Olympics 2020 Anomaly Detection Challenge, Zenodo (2019), doi:10.5281/zenodo.2629072 (CC BY 4.0).