Cosmos3-Nano

Mixture-of-Transformers architecture of Cosmos 3 (Figure 5 of the technical report). The understanding pathway reads the text prompt, the generation pathway reads the video latents and the actions, and the two meet in a shared attention.

NVIDIA Cosmos3-Nano in the LeRobot format. This is the base checkpoint for fine-tuning as a robot policy. Taking the camera images and a language instruction as input, the policy predicts the future video frames together with the next chunk of actions.

Fine-tuning

Install LeRobot from source with the Cosmos3 extras (the Cosmos3 policy is added in huggingface/lerobot#5134):

pip install -e ".[cosmos3,training]"

Full fine-tuning of Cosmos3 Nano uses FSDP2 over at least 4 GPUs of 80 GB, for example on LIBERO-10:

accelerate launch --num_processes=8 $(which lerobot-train) \
  --policy.type=cosmos3_nano \
  --policy.pretrained_path=lerobot/Cosmos3-Nano \
  --policy.load_action_modules=false \
  --policy.gradient_checkpointing=true \
  --policy.repo_id=${HF_USER}/Cosmos3-Nano-Policy-LIBERO_10 \
  --dataset.repo_id=lerobot/LIBERO_10-Cosmos3 \
  --parallelism.dp_shard=8 \
  --accelerator.fsdp.mixed_precision_policy.param_dtype=bfloat16 \
  --accelerator.fsdp.mixed_precision_policy.reduce_dtype=bfloat16 \
  --accelerator.fsdp.mixed_precision_policy.cast_forward_inputs=false \
  --batch_size=32 \
  --steps=16000 \
  --output_dir=outputs/train/cosmos3_nano_libero_10

• --policy.load_action_modules=false starts the action modules from their initialization, as the native post-training does.
• The cameras, the action dims and the frame rate come from the dataset.
• config.json keeps dtype: float32 for float32 master weights in training. For inference and evaluation pass --policy.dtype=bfloat16.

The Cosmos3 documentation gives the full fine-tuning recipes (multi GPU, LIBERO, DROID), the dataset requirements and the evaluation.

Fine-tuned checkpoints

Citation

@techreport{nvidia2026cosmos3,
  title  = {Cosmos 3: Omnimodal World Models for physical AI},
  author = {{NVIDIA Cosmos Team}},
  year   = {2026},
  url    = {https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf}
}

License

The NVIDIA model materials use OpenMDW 1.1. Retain the license and the notices of NOTICE when redistributing them. The NVIDIA model card describes the intended use and the limitations of the original model.