AgentRoboCritic-Muse-Glimmer-30B
AgentRoboCritic-Muse-Glimmer-30B judges whether an executed robot action achieved its stated local intent. It reads the task, the agent’s intent and command, and visual observations from around the action. It returns a structured judgment and, when the action fails, a failure category and a short explanation grounded in the observations.
Paper: https://arxiv.org/abs/2611.26613
Project page: https://adu2021.github.io/blog/AgentRoboCritic/
Using the critic
The example assumes a machine with enough memory for the 30B model.
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
checkpoint = "cx-cmu/AgentRoboCritic-Muse-Glimmer-30B"
processor = AutoProcessor.from_pretrained(checkpoint)
critic = AutoModelForImageTextToText.from_pretrained(
checkpoint, dtype=torch.bfloat16, device_map="auto"
).eval()
Use action_system.txt as the system message. The user message contains the task, the action's stated intent and command, and the ordered observations with their camera views and timing. Our evaluation used temperature 0 and a maximum of 1,024 output tokens.
The model returns four JSON fields. For example, a successful action has this form:
{"intent_success":"achieved","failure_mechanism":null,"m1_subcategory":null,"failure_reason":null}
For a failure, failure_mechanism gives the category and failure_reason describes the visual evidence. Grasp failures (M1) also receive an m1_subcategory. The system prompt defines the categories and the judgments available when the evidence is inconclusive.
Training
LoRA (rank 32) supervised fine-tuning, merged into the base weights, on 12,844 actions across 12 manipulation benchmarks, labeled by GPT-6 Astra. 2 epochs, batch size 32, learning rate 5×10⁻⁵ with cosine schedule.
Evaluation
On 400 human-labeled test actions (264 failed), from episodes held out of training:
| Critic | Outcome Acc. (%) | Failure Mode Acc. (%) | Overall Acc. (%) |
|---|---|---|---|
| GPT-6 Astra | 79.25 | 80.68 | 71.25 |
| Muse-Glimmer-30B (zero-shot) | 52.50 | 32.58 | 34.75 |
| AgentRoboCritic-Muse-Glimmer-30B | 76.25 | 74.24 | 67.75 |
License
Apache 2.0, as the base model, subject to the Muse Glimmer Usage Policy. Modified from the base model by LoRA fine-tuning.
Citation
@misc{du2026agentrobocritic,
title = {AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control},
author = {Yiyang Du and Yuxuan Kuang and Qitao Zhao and Yixuan Wang and Zhanqiu Guo and Yi Xu and Xin Li and Wenbin He and Liu Ren and Chenyan Xiong},
year = {2026},
eprint = {2611.26613},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2611.26613}
}