ACT for the cup, trained on all seven trossen-bin-picking-sim datasets for 90,000 steps

An ACT policy for a bimanual WidowX AI station that picks up the red cup and puts it in the clear tray. From the four cameras and the 14 joint positions it outputs 14 joint targets, 100 steps at a time. It doesn't read the instruction, so it's for the cup only.

• Data: the 17 cup episodes of the training split in each of the seven kiteml/trossen-bin-picking-sim-* datasets (119 episodes, 48,069 frames).
• Run: 90,000 steps at batch 8 on one A100, about 4 hours. These are the 90,000-step weights. kiteml/trossen-bin-picking-sim-act-cup-all is the same data and settings trained for 30,000 steps.
• Evaluation, closed-loop on a simulated copy of the station, 20 runs per test on the two held-out cup layouts: placed 8 of 20 under room light, 3 of 20 in low light, 4 of 20 in warm light and 3 of 20 with extra objects on the table. Every placement is on the layout where the left arm picks the cup up. On the same starts, kiteml/trossen-bin-picking-sim-act-cup-baseline, trained on the same episodes in baseline alone for 30,000 steps, placed it 8, 2, 0 and 8 times, and the 30,000-step all-seven policy 2, 2, 1 and 0 times.

The tests, the limits and every policy's runs are in kite-ml/trossen-bin-picking-sim-datasets.

Loads with LeRobot 0.6 or later:

from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("kiteml/trossen-bin-picking-sim-act-cup-all-90k")

Simulation only. It hasn't been tested on a real robot.

Trained on datasets derived from Trossen Robotics' public bin_picking dataset (Apache-2.0), with LeRobot's ACT (Apache-2.0). Made by Kite.