ACT_sim_handle_basket (RoboSynChallenge submission checkpoint)

ACT (Action Chunking with Transformers) checkpoint for the RoboSynChallenge handle_basket simulation task, re-packaged in the LeRobot 0.4.4 layout so that it loads without any legacy code path.

This checkpoint is the imitation-learning component of our preliminary-round submission. It is used both

• as a stand-alone ACT baseline for handle_basket, and
• as the action-chunk generator that the submitted hybrid policy hands over to after its analytic grasp/place stage.

Provenance

The migration is format-only: no weight is retrained, re-initialised or fine-tuned. model.safetensors is byte-for-byte the upstream tensor payload.

Why a key-restoration pass was necessary

LeRobot's official migrator renames feature keys with ".".join(parts[:-1]).replace("_", "."), which turns observation_images_cam_high into observation.images.cam.high instead of observation.images.cam_high. Those keys never match the policy features, so the image branch would silently stay un-normalised (a mid-grey image stays 0.5 instead of becoming roughly 0.066 after ImageNet normalisation).

The migrator was therefore followed by a pass that rewrites the state-file keys back to the real feature names declared in config.json, and by an assertion that normalisation actually applies:

image 0.5 -> ~0.066   (must NOT be ~0.5)

If you re-derive these files yourself, please re-run that assertion — a checkpoint that looks fine can score 0 % purely because of this key mismatch.

Files

Interface

Usage

from lerobot.policies.act.modeling_act import ACTPolicy
from lerobot.policies.factory import make_pre_post_processors

policy = ACTPolicy.from_pretrained("<this-repo>")
pre, post = make_pre_post_processors(
    policy_cfg=policy.config,
    pretrained_path="<this-repo>",
    preprocessor_overrides={"device_processor": {"device": policy.config.device}},
    postprocessor_overrides={"device_processor": {"device": "cpu"}},
)
batch = {
    "observation.state": state,                 # [1, 14] float32
    "observation.images.cam_high": img_high,    # [1, 3, 480, 640] float32 in [0, 1]
    "observation.images.cam_right_wrist": img_rw,
    "observation.images.cam_left_wrist": img_lw,
}
action = post(policy.select_action(pre(batch)))  # [1, 14]

A ready-made adapter that handles both this 0.4.4 layout and the legacy b883328 layout is shipped in the submission repository alongside this card.

Evaluation protocol

• Task: handle_basket, setting random (domain randomisation on)
• Episode limit: 500 environment steps (configs/handle_basket/random/gym_config.json)
• Success criterion (official): the milk carton is inside the basket, the basket is lifted and displaced more than 0.15 m along +y, and all of that holds for 55 consecutive steps.

Limitations

• Simulation only. This checkpoint was trained on the released simulated handle_basket demonstrations; it is not a real-robot policy.
• It is not a stand-alone solution for the whole task. On its own it performs the grasp/place/carry sequence at the base rate of the released ACT baseline; the submitted hybrid policy adds an analytic, RGB-only grasp-and-place stage in front of it.
• Training data, seeds and hyper-parameters are those of the upstream released checkpoint; only the on-disk format was changed here.