A 5.03M-parameter verifier that scores candidate action chunks of a frozen SmolVLA policy. At every decision step the policy samples K candidate chunks, the verifier predicts the probability of task success for each candidate, and the highest-scoring candidate is executed. The policy weights are not changed.
One model covers all 40 LIBERO tasks (libero_spatial, libero_object, libero_goal, libero_10); the task is given through the language input.
| File | Content |
|---|---|
model.safetensors |
Weights and input-normalisation buffers |
config.json |
Model configuration, input sizes, training summary |
verifier.py |
Model definition (PaperGate: dynamics MLP + state-attention classifier) |
features.py |
Feature extractors for the image, language and robot-state inputs |
load.py |
load_verifier, score_candidates |
example.py |
Loading and shape check |
LICENSE |
Apache License 2.0 |
| Module | Parameters | Description |
|---|---|---|
| Language projection | 61,504 | Linear 960 → 64 |
| Dynamics MLP | 3,927,972 | Input layer, 4 residual MLP blocks (width 512, inner width 808, LayerNorm, ReLU, dropout 0.1), predicts the change of the 26-d robot state for each of the 10 steps; changes are accumulated onto the current state |
| State-attention classifier | 1,039,493 | Learned positions, one self-attention layer (8 heads, width 256), FFN 512, learned-query pooling, head 256 → 128 → 64 → 1 |
| Total | 5,028,969 |
Inputs and predicted state changes are standardised with fixed statistics stored in the checkpoint (buffers, not trained).
| Input | Shape | Definition |
|---|---|---|
robot |
(26,) | 8-d proprioceptive state given to the policy, 9 joint positions, 9 joint velocities |
image |
(768,) | DINOv2 ViT-S/14 (facebook/dinov2-small) CLS features of the agent-view and wrist-view images, 384 each |
language |
(960,) | Mean of the SmolVLA language-token embeddings of the instruction |
actions |
(10, 7) | First 10 steps of a candidate action chunk |
Output: one logit per candidate; sigmoid(logit) is the predicted probability that the episode succeeds.
from load import load_verifier, score_candidates
model = load_verifier(device="cuda")
# robot (26,), image (768,), language (960,), candidates (K, 10, 7)
scores = score_candidates(model, robot, image, language, candidates)
chosen = candidates[scores.argmax()]
features.py contains the functions that produce image, language and robot from a LeRobot LIBERO observation and a loaded SmolVLA policy.
HuggingFaceVLA/smolvla_libero), 50-step chunks, the first 10 steps of the chosen candidate are executed.Success out of 400 episodes per evaluation seed.
| Evaluation seed | Raw policy | With verifier (K = 5) |
|---|---|---|
| 720000 | 267 (66.8%) | 310 (77.5%) |
| 750000 | 278 (69.5%) | 338 (84.5%) |
| 760000 | 268 (67.0%) | 314 (78.5%) |
| Total (1,200) | 813 (67.8%) | 962 (80.2%) |
By suite, three evaluation seeds pooled (300 episodes per suite), raw → verifier: libero_spatial 63.3% → 75.7%, libero_object 91.0% → 94.7%, libero_goal 71.7% → 87.7%, libero_10 45.0% → 62.7%.
| Item | Value |
|---|---|
| Data | 6,000 rollouts of the SmolVLA policy on the 40 tasks: 4,381 successful and 1,619 failed episodes |
| Labels | Episode outcome (success or failure) for every 10-step window of the episode |
| Loss | Binary cross-entropy on success + mean squared error of the predicted robot-state changes (weight 1) |
| Optimiser | Adam, learning rate 1e-3, weight decay 1e-4, one-cycle schedule with 5% warm-up, gradient clipping 5 |
| Batch size / steps | 1,024 / 20,000 |
| Validation | Episode-level split (10% of the episodes held out); the checkpoint with the highest mean per-task validation AUC is kept (evaluated every 2,000 steps) |
| Selected checkpoint | Step 20,000, validation AUC 0.941 |
| Training seed | 0 |
The training rollouts were collected over several rounds in which the policy was guided by earlier per-task verifiers, so the data contain both raw and verifier-guided behaviour.
Apache License 2.0 (see LICENSE).
Add the reference of the accompanying paper here.