pi05-weigh-apple-seedance100-v1

A vision-language-action (VLA) policy for a bimanual AgileX Piper arm setup, fine-tuned from the π0.5 (pi05_base) foundation model. Given camera images and a natural-language instruction, it outputs a 50-step chunk of end-effector actions for both arms.

The task it was trained on is weighing an apple — handling an apple and placing it on a kitchen scale. The specific training variant is recorded internally as weigh_apple_v3_leftmost_power_lit_display, i.e. the scene targets the leftmost scale, with its display powered and lit.

⚠️ This is a diagnostic checkpoint, not a production policy

The training manifest that produced these weights records training_scope: "exploratory_diagnostic" and production_policy_eligible: false, with a distribution-acceptance claim of local_robustness (robustness near the training distribution) rather than broad_deployment.

In other words, the project that trained this model did not clear it for deployment. It is published as a research artifact. Do not treat it as a validated controller. See Evaluation status.

Why this checkpoint exists

Collecting real teleoperated robot demonstrations is slow and expensive. This model is part of a sim2video line of work: instead of recording real demonstrations, trajectories are produced in simulation and then re-rendered into photorealistic video with a video generation model (here, Seedance), so the policy trains on imagery that looks like the real world without ever recording it. This checkpoint is the policy that comes out the other end of that pipeline.

Model at a glance

Inputs and outputs

Inputs

• Images — three independent camera views, each 224×224 RGB.
• Language instruction — a natural-language prompt, tokenized with the PaliGemma tokenizer, up to 200 tokens.
• Robot state — ⚠️ not used. This policy was trained with state conditioning disabled (--model.no-discrete-state-input; the training manifest records state_conditioning: none). The policy is vision- and language-only: it does not read joint encoders. A norm_stats.json with 14-dim state statistics ships with the checkpoint because the data pipeline computes it, but the model does not consume the state field.

Outputs

• An action chunk: 50 future timesteps × 14 active dimensions.
• The 14 dimensions are, per arm (left then right): 3 end-effector translation + 3 end-effector rotation + 1 gripper.
• The model's internal action vector is padded to 32 dimensions; dimensions 14–31 are padding and must be discarded.

Actions are normalized. You must un-normalize them with the shipped assets/.../norm_stats.json before sending anything to a robot.

Repository layout

.
├── params/                        # Orbax OCDBT checkpoint — the weights (~12 GB)
├── assets/
│   └── agentic-sim2video/
│       └── piper_weigh_apple_20260922_v3_seedance100_inline224_v1/
│           └── norm_stats.json    # action/state mean, std, q01, q99 — REQUIRED
└── _CHECKPOINT_METADATA

params/ is a JAX/Orbax checkpoint, not safetensors — it will not load with transformers. You need openpi.

Note: only inference weights are published. The optimizer state (train_state/, ~31 GB) was not uploaded, so you can run this policy but cannot resume training from step 49999.

Usage

from huggingface_hub import snapshot_download

ckpt = snapshot_download("xiwenyoumu/pi05-weigh-apple-seedance100-v1")

Then load it with openpi, using the training config this checkpoint was produced under:

from openpi.training import config as _config
from openpi.policies import policy_config

cfg = _config.get_config("pi05_piper_dual_weigh_apple_eef_xyz3d_100")
policy = policy_config.create_trained_policy(cfg, ckpt)

action_chunk = policy.infer({
    # three 224x224x3 uint8 views — key names must match your config's data transform
    "images": {...},
    "prompt": "weigh the apple",
})["actions"]          # (50, 32) -> use [:, :14], then un-normalize

The config name pi05_piper_dual_weigh_apple_eef_xyz3d_100 refers to a project-local openpi config that is not part of upstream openpi. To use this checkpoint you will need an equivalent config declaring: pi05=True, action_dim=32, action_horizon=50, max_token_len=200, discrete state input disabled, and a Piper bimanual EEF policy transform over three camera views.

Training details

Training data — an internal dataset, agentic-sim2video/piper_weigh_apple_20260922_v3_seedance100_inline224_v1: simulated bimanual apple-weighing trajectories re-rendered into photorealistic video with Seedance, at 224×224, three views. This dataset is not public.

Final training loss — ≈0.00050 (flow-matching loss at the last logged step). Per-group losses were highest on left-arm rotation (≈0.0019) and right-arm rotation (≈0.0013), lowest on translation and gripper dimensions (≈0.0005–0.0008). Loss was flat across the action horizon (early ≈0.00052, middle ≈0.00048, late ≈0.00051).

Evaluation status

Read this before using or citing this model.

• ❌ The project marked this checkpoint as not production-eligible. The manifest records training_scope: "exploratory_diagnostic" and production_policy_eligible: false.
• ❌ No held-out evaluation was run. The periodic "eval" during training was computed on a fixed first training batch with a fixed RNG — the training manifest records it verbatim as "fixed first transformed batch with fixed RNG; not held-out". It is a numerical-stability probe, not a measure of generalization.
• ❌ No real-robot success rate is reported here. Nothing in this repository demonstrates that the policy weighs an apple on physical hardware.
• ❌ No simulation benchmark score is reported here.
• ⚠️ Distribution-acceptance claim is local_robustness, not broad_deployment — the upstream data-acceptance step only asserted robustness near the training distribution.
• ✅ What is verified: training ran to completion for 50,000 steps with finite loss and gradients throughout, and the loss converged to ≈0.00050.

Low training loss on generated video says the policy fits its training distribution. It does not establish sim2real transfer.

Limitations and intended use

• Diagnostic artifact. Published for research inspection and reproduction, not for deployment. See the banner above.
• Single task, single embodiment. Trained only on apple-weighing with a specific bimanual Piper setup, specific camera placement, and 224×224 three-view input. It is not a general-purpose manipulation policy and should not be expected to follow arbitrary instructions.
• Narrow scene variant. The training scene targets one specific configuration (leftmost scale, powered lit display). Behaviour on other scale positions, unlit displays, or other objects is untested.
• Trained on generated, not real, video. Systematic artifacts of the video generator are baked into what the policy expects to see. Real-world appearance gaps are an open risk.
• No state feedback. Being vision-only, the policy cannot recover from a mismatch between where it believes the arms are and where they actually are.
• Safety. This model drives physical robot arms. Any real-hardware deployment needs workspace limits, velocity/force limits, and an accessible emergency stop. Do not run it on hardware near people without those safeguards.

Related

• xiwenyoumu/pi05-beat-drum-seedance100-v1 — same pipeline, same hyperparameters, drum-beating task.

License

Released under Apache-2.0, following the license of the upstream openpi π0.5 base model it was fine-tuned from.

Acknowledgements

Built on openpi and the π0.5 base model by Physical Intelligence.