CaPtain-4B

CaPtain-4B is the final model of CaPtain: A Pipeline for Post-Training Code-as-Policy Robot Agents. It is Qwen3.5-4B after supervised fine-tuning (SFT) and reinforcement learning (RL). It is a code-as-policy agent: it controls a robot by writing Python in CaPtain-Harness, reads the receipts of its own robot calls, and decides itself when the task is done.

What the model is

• A code-writing agent, not a VLA. The model never outputs actions or joint targets directly. At every request it writes Python for the harness's single python tool: a persistent, sandboxed Python 3.11 session. The robot API in that session has 22 low-level capabilities (perception, geometric measurement, IK-level guarded motion) plus finish(). There are no grasp or place macros.
• What it sees: camera images, its own code and output, and a one-line text receipt for every robot call.
• What it never sees: object poses, contacts and the success predicate. These stay on the host.
• Perception runs as services. The harness's segment and mark_point tools call SAM 3 and Molmo2-8B, and positions come from depth back-projection. The model calls these tools; it does not replace them.
• Format. bf16 safetensors with the Qwen3_5ForConditionalGeneration architecture. The lm_head is untied. The tokenizer, processor and chat template are the same as Qwen3.5-4B. During SFT the vision tower was kept frozen.

How to use

The model only works well inside CaPtain-Harness (see Prompt format). Serve it with vLLM as an OpenAI-compatible endpoint, start the perception services, and run episodes with the harness.

# 1. policy server (vLLM 0.29, OpenAI-compatible; these are the flags used for the paper's evaluations)
vllm serve YijiaFan/CaPtain-4B --served-model-name captain-4b --dtype bfloat16 --max-model-len 81920 \
  --enable-prefix-caching --limit-mm-per-prompt '{"image":12}' --max-num-seqs 16 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml --port 8600
#    (the harness wraps this command as: services/serve_policy.sh YijiaFan/CaPtain-4B 8600)

# 2. harness, perception services and one LIBERO episode
git clone https://github.com/waltstephen/CaPtain-Harness.git && cd CaPtain-Harness
#    install as in INSTALL.md (LIBERO venv + the agent's sandboxed Python 3.11), then:
services/start_perception.sh                                    # SAM 3 :8114 + Molmo2-8B :8122
export POLICY_URL=http://127.0.0.1:8600/v1 STUDENT_PYTHON=~/envs/student-py311/bin/python EGL_DEVICE=0
eval/run_episode.sh libero_spatial 0 11 runs/demo
cat runs/demo/libero_spatial_0_seed-11/episode_result.json      # host_final_success, agent_finished, ...

# 3. a whole benchmark, scored as in the paper
python3 eval/run_jobs.py eval/tasks/libero_official_240.tsv --out runs/captain-4b --parallel 8
python3 eval/summarize.py eval/tasks/libero_official_240.tsv runs/captain-4b

Evaluation protocol used for every number below (the defaults of eval/run_episode.sh):

• sampling T = 1.0, top_p = 1.0, thinking on;
• budgets per episode: 60 model requests, 1,080 s wall time and 150 physical actions;
• context: a 64k-token window with the harness's compaction.

The task lists for LIBERO-PRO, LIBERO-Plus, LIBERO-MAX and the transfer benchmarks are in the harness (eval/tasks/, eval/transfer/).

Prompt format

CaPtain-4B was trained on the harness's exact request template:

• the system prompt and environment context the harness sends;
• TASK.md, inlined in the system prompt;
• the python tool schema (python-runtime/v1 contract);
• receipts, context compaction notes and image placement.

The model uses the standard Qwen3.5 chat template, with <think> reasoning and XML tool calls, but it expects this harness-specific content. Do not hand-write prompts for it: run it through CaPtain-Harness, and do not change model-visible text (TASK.md, tool descriptions, receipts, request template) when evaluating it. Outside the harness it behaves like an ordinary small VLM, with no robot ability.

Results

All numbers are success rates (%) in simulation under the protocol above.

• Host success: the benchmark's hidden goal predicate holds at the end of the episode.
• Strict success: host success, and the agent also called finish() itself.
• The LIBERO row uses the held-out official initial states 11–16 of all 40 tasks of the four standard suites. RL never used these states.
• LIBERO-PRO, LIBERO-Plus and LIBERO-MAX were never used in training.

LIBERO family (host / strict):

Zero-shot transfer (RL used only the 40 LIBERO tasks):

The transfer backends give the agent inputs beyond the benchmarks' official policy observation: simulated depth, camera calibration, a wrist force or contact signal, and on SimplerEnv an added wrist camera. They are listed per benchmark in the harness (eval/transfer/README.md). The agent never receives object poses, the success predicate or task progress. Reference points for each benchmark are in the paper and the CaPtain README.

Training

1. Base. Qwen3.5-4B, with the lm_head untied from the embeddings.
2. SFT → CaPtain-4B-SFT. Trained on CaPtain-SFT-Data:
   • synthetic agent trajectories collected in CaPtain-Harness, plus visual QA at loss weight 0.5;
   • only assistant turns are supervised, the vision tower is frozen, and a text-hint mask (p = 0.2) hides earlier pixel coordinates;
   • 2,435 steps (1.5 epochs), peak LR 1e-5 with a cosine schedule, DeepSpeed ZeRO-3, bf16.
3. RL → CaPtain-4B. On-policy RL in LIBERO with a simulator critic:
   • A small critic (5 MLPs) reads the simulator's physical state: object poses, grasp state and sub-goal progress. The change in its predicted success across each model request becomes that request's credit. The policy never sees this state. The critic starts from an offline warm start and is co-trained online.
   • Advantages: GAE (λ = 0.9) on the critic, combined with a group-relative outcome term (G = 4 rollouts per start).
   • Loss: SAO-style with a double-sided ratio mask, plus a k3 KL term (β = 0.002) to CaPtain-4B-SFT.
   • Batch: 256 episodes per round (64 starts × G = 4); K = 4 rounds per optimizer update, so 1,024 episodes per update.
   • CaPtain-4B is round 300 = 75 optimizer updates (76,800 training episodes), on wide-layout LIBERO start states with randomised lighting, camera pose and materials.

The full configurations are in the CaPtain repository (sft/configs/captain-4b-sft.yaml, rl/configs/captain-4b.json) and in CaPtain-artifacts.

Limitations

• Simulation-trained. RL used only the 40 LIBERO tasks. The SFT data are simulated episodes, almost all on LIBERO (see the dataset card). Transfer to other benchmarks is weaker than in-domain performance.
• Missed finishes. Sometimes the scene is solved but the agent does not call finish(). This explains most of the gap between host and strict success.
• Hard tasks. Failures concentrate in long-horizon and articulated tasks (LIBERO-10, some LIBERO-Goal tasks).
• Two-arm coordination. The policy acts with one arm at a time. On RoboTwin, tasks that need simultaneous two-arm motion fail. Reach limits of unfamiliar arms are the other common transfer failure.
• Inputs. The model relies on the harness's perception services (SAM 3, Molmo2-8B), depth and camera calibration. The transfer backends add the inputs listed above.
• Real robots need adaptation. The harness has a hardware backend (driver protocol, kinematic twin, cameras), but this checkpoint was not trained on real-robot data. The paper's real-robot episodes used an embodiment-adapted checkpoint. Running the model on hardware needs your own driver, calibration and safety guards.
• Interface ceiling. The agent is bounded by the 22-capability API and by what the SFT trajectories demonstrate. For example, sustained pressing (CaP-Bench Wipe) cannot be expressed with the guarded motion API.
• Research artifact. Not validated for safety-critical use or for unsupervised use on physical robots.

License

Apache-2.0, the license of the base model Qwen3.5-4B. When you run the model together with other components (LIBERO, SAM 3 under the SAM License, Molmo2, simulators), their licenses also apply; see THIRD_PARTY_LICENSES.md in the CaPtain repositories.

Citation

@article{fan2026captain,
  title   = {CaPtain: A Pipeline for Post-Training Code-as-Policy Robot Agents},
  author  = {Fan, Yijia and He, Xilin and Jiang, Yilei and Wu, Dongming and Wang, Jiaxu and Yue, Xiangyu},
  journal = {arXiv preprint},
  year    = {2026}
}

Contact: Xiangyu Yue (xyyue@ie.cuhk.edu.hk), MMLab, The Chinese University of Hong Kong.