Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation

ViGAR: Visual Goal-conditioned Action Reasoning

Shukai Gong^1* · Xuanran Zhai^2* · Yintianrun Zhang^1* · Ruopeng Cui² · Ye Huang¹ · Yiyang Fu¹ · Dexuan Lyu²
Chaojie Li² · Xinyi Song² · Peiwen Lin² · Chuang Wang² · Mingyuan Jia³ · Yufan Deng¹
Jiaxin Fang³ · Bo Liang¹ · Jiaxin Li¹ · Yuxiang Gao^3† · Hao Liu^2† · Daquan Zhou^1†

¹ Peking University ² AgiBot ³ CocoMatrix

^* Equal contribution ^† Corresponding author

🔥 News

2026.10.11: ViGAR ranks #3 on the RoboTwin 2.0 Leaderboard.
2026.10.05: ViGAR is released on arXiv.

📑 Todo List

Evaluation code and checkpoints on RoboTwin.
Training code and data on RoboTwin.
900-hour real-robot dataset with subtask annotations.

🎬 Demo

⚙️ Getting Started

git clone https://github.com/DAGroup-PKU/ViGAR.git
cd ViGAR

Step	Guide
Set up the environments	Installation
Download weights and data	Download
Evaluate on Simulation Benchmark	Evaluation
Train the policies and subgoal planners in ViGAR	Training

💪 Repository

vigar/            Policy model, training and serving
subgoal_planner/  Planner runtime and training implementation
simulation/       RoboTwin tasks and simulator setup
data/             Annotation, goal generation and action conversion
evaluation/       Policy/planner services and closed-loop evaluation
scripts/          Installation and launch commands
configs/          Task and inference settings
docs/             Installation, download, evaluation and training guides

The policy uses VeOmni with a bundled Cosmos runtime. Planner processes select their own Cosmos runtime through PYTHONPATH; they run separately from policy and simulator processes.

🙏 Acknowledgements

Built on NVIDIA Cosmos, VeOmni, RoboTwin, Wan2.2 and Qwen3-VL.

✏️ Citation

@article{gong2026rethinking,
  title={Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation},
  author={Gong, Shukai and Zhai, Xuanran and Zhang, Yintianrun and Cui, Ruopeng and Huang, Ye and Fu, Yiyang and Lyu, Dexuan and Li, Chaojie and Song, Xinyi and Lin, Peiwen and others},
  journal={arXiv preprint arXiv:2610.02368},
  year={2026}
}

License

Original project code uses the MIT license. Bundled components retain their upstream licenses. Cosmos-derived model materials use OpenMDW-1.1.

DAGroup-PKU/ViGAR