RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning

RoboAtom-Anno · 9B and 27B video annotation models

Models · Download · Inference · Output and evaluation · License and citation

Overview

RoboAtom-Anno predicts atomic actions from robot videos and task instructions. Each predicted segment contains a skill label, an action description, and frame boundaries. The release provides merged 9B and 27B weights, with tokenizers, video processors, and generation settings.

Models

Use Python 3.11+, transformers==5.8.0, and compatible CUDA-enabled PyTorch. The RoboAtom inference implementation handles synchronized camera composition, frame sampling, and frame-index prompts. GPU requirements depend on the model and video input.

Download

These examples use ./model for weights and ./data for the benchmark:

hf download InternRobotics/RoboAtom --repo-type model \
  --include 'RoboAtom-Anno-9B/**' --local-dir ./model
hf download InternRobotics/RoboAtom --repo-type dataset \
  --include 'RoboAtom-Bench/**' --local-dir ./data

Replace RoboAtom-Anno-9B with RoboAtom-Anno-27B to select the larger model. The Hugging Face CLI (hf) is required for these download commands.

Inference

1. Install the code

From your RoboAtom code checkout:

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[inference]'

2. Start the model service

The following paths match the download layout above; replace them if you chose another location:

roboatom serve \
  --model ./model/RoboAtom-Anno-9B \
  --media-root ./data/RoboAtom-Bench

The service listens on http://127.0.0.1:3000. Change the model directory to select the 27B weights.

3. Generate predictions

In another terminal with the same environment activated:

roboatom infer \
  --input ./data/RoboAtom-Bench/benchmark.jsonl \
  --output outputs/roboatom_predictions.jsonl \
  --base-url http://127.0.0.1:3000

Choose a new output filename for each run; the inference client refuses to overwrite an existing prediction file.

Output and evaluation

The client writes one prediction record per benchmark input, with an identity key and ordered action segments. A segment covers start_frame through end_frame inclusively; frames 0–74 contain 75 frames. Keep ground truth separate from model inputs.

From the code checkout, score the predictions:

roboatom evaluate \
  --gt ./data/RoboAtom-Bench/gt.jsonl \
  --benchmark-input ./data/RoboAtom-Bench/benchmark.jsonl \
  --prediction outputs/roboatom_predictions.jsonl \
  --protocol configs/roboatom_bench.yaml \
  --output-dir outputs/roboatom_evaluation

The scorer writes metrics.json, summary.tsv, and episodes.jsonl. See the benchmark card for data fields and the code README for the evaluation protocol.

License and citation

The model directories contain the 9B Apache-2.0 license and 27B Apache-2.0 license. See the project's third-party notices for component attribution and citation file for publication metadata.

If you use RoboAtom in your research, please cite:

@misc{wei2026roboatomlargescaleatomicactionsupervision,
      title={RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning},
      author={Xueyuan Wei and Xuanxu Wang and Xudong Li and Yuping He and Wanshui Gan and Jiaqi Peng and Wenqi Guo and Jianjun Zhou and Junting Dong and Chunhua Shen and Haoyu Guo and Mulin Yu and Tao Lu},
      year={2026},
      eprint={2610.40207},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2610.40207},
}