SyncWorld three-camera strip384, recent8 (baseline), update 30000

Status: final checkpoint (update 30000 of 30000).

One of the two runs of SyncWorld-dev's three-camera comparison (in-context branch, wm_ttt/retrieval/README.md "Three-camera comparison", upstream commit 40be701 plus local fixes): recent8 (this repo, the baseline) against the simplified nearest retrieval with joint camera views, views first, at 10 deg / 10 cm (yyuncong/SyncWorld-mv3-strip384-views-first-v10-step*). Run qctx_incontext_mv_A_recent8_8gpu_4n30k_s384, recipe wm_ttt/retrieval/recipe_qctx_incontext_mv_A_recent8_8gpu.toml (training/recipe.toml).

• Frames: every frame is a 1536x384 strip of three 512x384 (4:3) cells: the recording's two third-person cameras, shuffled between the first two cells per stream, and the wrist camera in the last, with per-stream view dropout (one camera 10 %, two 5 %) and random vertical crops (third person 0.3-0.65, wrist 0.5-1.0), as cosmos-framework-dev's strip384 run. One tiling plan per stream holds for the query and all of its chunks.
• Layout: full-attention in_context, no TTT memory. A query is one anchor frame plus its own 16 future commanded actions, and the model generates the next 16 frames; before it, K observed 16-transition video/action chunks (K uniform over 1..8, one K per update for all GPUs) sit in the same attention window as clean conditioning.
• recent8: the K chunks are the query's own K most recent chunks.

Training

• Initialization: cosmos-framework-dev's strip384 run (gripperhead-mv-strip384-4x3-fork174k-4x8, iter 43500), already trained on these four datasets as 1536x384 strips.
• Optimization (the recipe as committed): AdamW (betas 0.9 / 0.99, weight decay 0.05), LR 5e-5 after a 400-update linear warm-up, then constant; trains moe_gen, the time embedder and the VAE/action projections; one action-axis relabel per stream (p 1.0); bf16 compute with fp32 master weights for the optimizer; full activation checkpointing.
• Scale: 4 nodes x 8 H100 (HSDP: an 8-way FSDP group per node, 4 replicas), 1 stream per GPU = 32 streams per update; 30,000 updates; a checkpoint every 1000.
• Data (syncworld_ttt/data/mv_strip_v2, media streamed from the shard sets): robocasa_v2 10,478 recordings (the formal_2 reference units only), robomimic_v2 16,608, rlbench_wrist_mv 11,314 (1,061 distinct trajectories over 8 render sets), droid 68,768 (recordings over 3,000 frames dropped); expert recordings only; component weights robocasa 2 : robomimic 2 : rlbench 1 : droid 2.
• Compute: MAST (spatial_ai_research), conda cosmos_fdm_conda_rlr:98a0b25c.

Training loss

train@2/loss (the query's video flow-matching loss) averaged over the 500 updates before each mark:

Files

• model/: the trainer's DCP model state, cast to bf16 (every tensor checked after reload).
• config.pkl: the trainer's config.
• training/recipe.toml: the recipe.