Flex-π · RoboDojo_real (ARX X5, Piper, Piper X)

Three Flex-π checkpoints for the RoboDojo_real robots, one per robot. Each is fine-tuned from flex-pi/flexpi-pretrain-preview on that robot's RoboDojo_real demonstrations and is served through XPolicyLab's FlexPi policy adapter. The model reads three RGB cameras plus the joint and end-effector state of both arms, and returns 32-step action chunks. It trains on RGB + DINO modalities and reads no depth.

Checkpoints

Each folder holds three files:

• step_N.pt: the weights, bf16, 12,082,213,237 bytes.
• config.yaml: the architecture and processor, which the adapter reads at boot. The file is the same in all three folders.
• dataset_stats.json: that robot's normalization statistics.

Model

Serving

The serving code is XPolicyLab's FlexPi adapter, from its pull request: branch flexpi-robodojo-real of github.com/YOLO12138/XPolicyLab until it is merged (checked at commit 1f0eb990: with these folders its answers are bit-identical to the offline checks below). The adapter is policy/FlexPi; its README.md section RoboDojo_real checkpoints is the full deploy reference.

Requirements

• Code: that XPolicyLab as the XPolicyLab/ directory of a RoboDojo checkout; the server reads env_cfg/ from the parent of XPolicyLab/. bash XPolicyLab/policy/FlexPi/install.sh flexpi creates the conda environment (python 3.10, torch 2.7.1+cu128).

• GPU: about 16 GB of memory per server, with the T5 text encoder on the CPU.

• Base files under policy/FlexPi/FlexPi/checkpoints/, the default of DIFFSYNTH_MODEL_BASE_PATH: the Wan2.2 VAE, the umt5-xxl text encoder and its tokenizer (about 12.8 GB, from ModelScope). config.yaml sets skip_dit_load_from_pretrain: true, and the checkpoint carries every fine-tuned weight, so the server does not read the Wan2.2-TI2V-5B DiT or the ActionDiT init file.

  cd <RoboDojo root>/XPolicyLab/policy/FlexPi && conda activate flexpi
  export DIFFSYNTH_MODEL_BASE_PATH="$(pwd)/FlexPi/checkpoints"
  python - <<'EOF'
  from flexpi.models.helpers.io import ModelConfig
  for repo, pattern in [
      ("DiffSynth-Studio/Wan-Series-Converted-Safetensors", "Wan2.2_VAE.safetensors"),
      ("DiffSynth-Studio/Wan-Series-Converted-Safetensors", "models_t5_umt5-xxl-enc-bf16.safetensors"),
      ("Wan-AI/Wan2.1-T2V-1.3B", "google/umt5-xxl/"),
  ]:
      ModelConfig(model_id=repo, origin_file_pattern=pattern).download_if_necessary()
  EOF
  conda deactivate
  

• DINOv3 timm/vit_base_patch16_dinov3.lvd1689m, fetched from the Hugging Face hub at boot. An offline host needs it in the local HF cache.

• env_cfg/<env_cfg_type>.yml in the RoboDojo root. Its config.robot names an entry of env_cfg/robot/_robot_info.json with arm_dim [6, 6] and ee_dim [1, 1]. RoboDojo's open-source env_cfg has arx_x5.yml only; without a piper.yml or piper_x.yml, the server stops at init with FileNotFoundError.

Download

cd <RoboDojo root>
git -C XPolicyLab fetch https://github.com/YOLO12138/XPolicyLab flexpi-robodojo-real && git -C XPolicyLab checkout 1f0eb990
cd XPolicyLab/policy/FlexPi
hf download flex-pi/flexpi-robodojo-real --include "arx_x5/*" --local-dir checkpoints/flexpi-robodojo-real

hf comes with huggingface_hub 0.34 or later; inside the flexpi environment (huggingface_hub 0.29.2) the same command is huggingface-cli download. --local-dir writes regular files. The symlinked layout of the HF cache does not serve these checkpoints; the launcher refuses it at start.

Launch

One server per robot, each from a shell at the conda base environment:

cd <RoboDojo root>/XPolicyLab/policy/FlexPi
bash setup_eval_policy_server.sh RoboDojo_real <task_name> checkpoints/flexpi-robodojo-real/arx_x5 \
    arx_x5 joint 0 <gpu_id> flexpi <port> 0.0.0.0 2>&1 | tee flexpi_server_arx_x5.log

The positional arguments are bench_name task_name ckpt_name env_cfg_type action_type seed gpu_id conda_env port host; task_name only labels the run, and env_cfg_type names the folder's robot. The launcher activates conda_env itself, under set -u; in a shell where that environment is already active, a conda deactivate hook that reads an unset variable can stop it right after its [SERVER] lines.

The PR's deploy.yml ships the settings these checkpoints need:

The server stops at boot when the folder's statistics name another robot than env_cfg_type, or when eef_frame is not relabeled. A correct start logs these lines:

• [SERVER] trained config: .../flexpi-robodojo-real/arx_x5/config.yaml
• [FlexPi] stats coverage robodojo_realworld_34d_relabeled_arx_x5: eef_frame=relabeled env_cfg_type=arx_x5 match
• [FlexPi] Loaded model + processor from trained config: .../arx_x5/config.yaml
• [FlexPi] ready: ... needs_depth=False state_bridge=yam32 eef_frame=relabeled

The station's client also needs policy/FlexPi: it reads its deploy.yml and runs its episode loop, deploy.py.

Robot side

Observation, every control tick

• Left and right are the robot's own left and right; the left arm appears in the left half of the head image.
• The EE pose is each arm's link6 (flange) point in that arm's own frame, as the RoboDojo_real recordings store it. There is no shared world frame.
  • arx_x5: the origin is that arm's link6 position at zero joints, and the axes are parallel to the arm base (x forward, y left, z up).
  • piper, piper_x: the arm's base frame (x forward, y left, z up).
• The adapter turns the recorded orientation into the model's tool frame itself (eef_frame: relabeled), so the pose is sent unconverted.
• The quaternion is normalized and turned into a rotation matrix, so its sign does not matter.
• The state's positions, joints and grippers are clamped to their trained q01..q99 range before the model sees them, so an out-of-range reading raises no error.
• The recordings start with all joints near 0 and both grippers open (97–100 % of episodes). At that pose a robot whose conventions match the recordings reads about these values; the quaternion's overall sign is free:

Output

Each call returns 32 targets, one per control tick. Element k is the target for tick t+k+1 after the observation at tick t. XPolicyLab's client loop executes all 32 before the next call.

• joint:
  • left_arm_joint_state, right_arm_joint_state (6): absolute joint angles in rad, the current joints plus the predicted deltas. The adapter applies no joint-limit or velocity clipping.
  • left_ee_joint_state, right_ee_joint_state (1): the grippers.
• ee:
  • left_ee_pose, right_ee_pose (7): [x, y, z, qw, qx, qy, qz] in the same recorded per-arm frame as the observation, with qw ≥ 0. The point is the link6 flange, not a fingertip.
  • The same two gripper keys.
• Both action types decode one network output.
• The gripper target is the predicted gripper reading of the next tick, on the recorded scale, clipped to [0, 1] (the Pipers' recorded readings reach -0.04). The recordings carry no separate command: each recorded action equals the next recorded state.

Frames

The model's tool frame has z along the approach axis (wrist to fingers), y along the finger-opening axis and x pointing away from the wrist camera. At the start pose this is z forward, y left, x down on all three robots, and wrist roll turns about z. The adapter maps the recorded orientation R onto it with one constant rotation P per robot, R_model = R P; ee outputs go back through R P^T. Positions, joints and grippers pass through unchanged.

XPolicyLab's _EEF_REFRAME_P_ROBODOJO_REAL, used by other policies, is a different table: P followed by a 180° roll. A pose that arrives with P already applied gets it twice: arx_x5 and piper_x then sit 90° off (piper's P is the identity), and nothing raises.

Cameras

• cam_head, cam_left_wrist and cam_right_wrist are all required, as RGB uint8.
• The recordings are 480×640 JPEGs in XPolicyLab's legacy channel-reversed byte format. decode_image_bit returns them as RGB, which is what the model trained on.
• A non-uint8 array is cast to uint8, so a float image in [0, 1] arrives black.
• The adapter resizes as training did, 480×640 to 256×320, then to 224×224 for the wrist cameras. Another aspect ratio is distorted differently from training.
• In the recordings each image trails the state by about 2 frames: about 1.5–1.7 frames for the wrist cameras, and about 1.9 (arx_x5) or 2.6–2.8 (Pipers) for the head camera. The model trained on that pairing.

Instructions

The instruction goes in alone. The adapter wraps it in the training template. A missing instruction becomes complete the task, and a bytes value becomes the text b'...'; neither prompt was trained.

Timing

• Control rate: 30 Hz (arx_x5), 30 Hz (piper), 25 Hz (piper_x), the recorded rates. A 32-step chunk spans 1.07 / 1.07 / 1.28 s. The model has no rate input, so at another rate the learned motions play back faster or slower.
• Latency on one A100 (bf16, 10 denoising steps): about 1.0 s per chunk. The first call after a reset or an instruction change takes about 25 s more, for the umt5-xxl encode on the CPU; the encoded prompt is cached until the next reset.
• Boot: about 2–3 minutes.

Validation

All checks ran offline through the FlexPi adapter with the settings above; the PR at 1f0eb990 reproduces them bit for bit.

• Inputs: the adapter's state built from the raw recorded values equals the training state exactly, on every training frame. Its image inputs are identical to the training inputs.
• Output decoding: the joint and ee paths turn the recorded actions back into the recorded targets within 1.2e-7 rad and 3.2e-8 m.
• Weights: every transformer weight loads bit-exactly. With only the VAE, T5 and tokenizer as base files, these folders give outputs bit-identical to the training run's own config served with every base file (one window per robot, CPU, fp32).
• Serving: downloaded with the command above and started with the launch command, the server's answer over XPolicyLab's websocket client equals the direct replay bit for bit (arx_x5, CPU, fp32).
• Replay: 24 recorded windows per robot (4 per task) on an A100 in bf16, each 32-step chunk compared with the recorded future. "Hold" repeats the current state.

The replayed windows come from the training demonstrations, since every recording was used for training. Behaviour on the real robots is untested.

License

The weights are released under the MIT license. The fine-tuning data, RoboDojo, is Apache-2.0; the base model's training data carries the terms listed on the flexpi-pretrain-preview card. At inference the model downloads DINOv3 timm/vit_base_patch16_dinov3.lvd1689m, which is under the DINOv3 License, and Wan2.2 components, which are Apache-2.0.

Citation

@article{yan2026flexpi,
  title   = {Flex-$\pi$: A Multi-Stream World-Action Model with Compute Flexibility},
  author  = {Yan, Ge and Liu, Jinghao and Fan, Yuzhi and Cai, Lei and Liao, Minwen
             and Zhang, Jesse and Fox, Dieter},
  journal = {arXiv preprint arXiv:2608.10860},
  year    = {2026},
  url     = {https://arxiv.org/abs/2608.10860}
}