RoboBeats-4B

📄 Paper | 💻 Code | RoboBeats-0.8B · RoboBeats-2B · RoboBeats-4B

RoboBeats is a small, strictly causal VLM gate from Learning to Live: A Step Toward Lifelong Embodied Intelligence. It watches a 1 fps video stream and scores every frame with the probability that a new event has just occurred. Only the frames around detected events are written into memory.

This is a LoRA adapter for Qwen/Qwen3.5-4B; the base model is downloaded automatically.

Usage

Install the code repository, then from its root:

python -m robobeats my_video.mp4 --gate-model superbigsaw/RoboBeats-4B --out probs.json

from robobeats.gate import Gate, select
from robobeats.video import iter_ticks

gate = Gate('superbigsaw/RoboBeats-4B')
probs = gate.stream((t, im.convert('RGB')) for t, _, im in iter_ticks('my_video.mp4', 0, 600))  # [(second, p)]
kept = select(probs, start=0)  # an event opens at p > 0.5 and closes at p < 0.3

The score at second t uses only frames up to t. Long streams are processed in chunks of 100 frames with 10 frames of overlap.

Citation

@article{che2026learning,
  title         = {Learning to Live: A Step Toward Lifelong Embodied Intelligence},
  author        = {Che, Lirong and Wang, Peidong and Su, Yimeng and Wang, Jiong and Chen, Yang and Fu, Wenbo and Huang, Zhenyu and Xu, Tiancheng and Yang, Yuzhe and Xie, Dongli and Cao, Xu and Wang, Chuang and Liu, Daqi and Su, Jian},
  journal       = {arXiv preprint arXiv:2611.00180},
  eprint        = {2611.00180},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  year          = {2026},
  url           = {https://arxiv.org/abs/2611.00180}
}