RealtimeWAM:
One-Step Asynchronous World Action Models

💡 Why RealtimeWAM

• 🥇 Pioneer work: The first one-step World Action Model (WAM) with near-lossless performance, built on any Mixture-of-Transformers (MoT)-based WAM architecture.
• 🏆 Superior performance: Maintains less than 1% average accuracy drop compared with multi-step counterparts across diverse benchmarks, including RoboTwin 2.0, LIBERO, and LIBERO-Plus.
• ⚡ Extreme acceleration: With one-step distillation, asynchronous inference, CUDA Graph, and efficient kernels, achieves 24.55× end-to-end speedup on Fast-WAM (12.2 ms) and 13.56× on Faster-WAM (16.1 ms) on a single NVIDIA H100, enabling real-time action generation. Latency includes VAE encoding and excludes text encoding, which is performed once per episode.

💪 TODO

^* and ^† denote variants based on Fast-WAM and Faster-WAM, respectively.

✨ Quick Start

Environment

Inference

RealtimeWAM uses the LightX2V environment. You can follow the environment setup instructions in the LightX2V README. For comprehensive usage instructions, please refer to the LightX2V documentation: English Docs | 中文文档. We highly recommend using the Docker environment, as it is the simplest and fastest way to set up the environment. For details, please refer to the Quick Start section in the documentation. If you prefer to build from source, please refer to Building from Source in the LightX2V README.

Evaluation

We provide uv-based environment setup scripts for LIBERO, LIBERO-Plus, and RoboTwin 2.0. We recommend CUDA 12.8 and PyTorch 2.7.1. The scripts create independent Python 3.10 environments with LightX2V and PyTorch 2.7.1, and initialize the corresponding benchmark submodules. Run the following commands from the LightX2V repository root. Install only the environment you need.

For LIBERO and LIBERO-Plus, install and activate their shared evaluation environment:

bash scripts/bench/robotics/install_env.sh libero
source .venvs/libero/bin/activate

For RoboTwin, first install CUDA Toolkit 12.8 and set CUDA_HOME to its installation path, then install and activate the evaluation environment:

export CUDA_HOME=/usr/local/cuda-12.8
export PATH="$CUDA_HOME/bin:$PATH"
bash scripts/bench/robotics/install_env.sh robotwin
source .venvs/robotwin/bin/activate

Dependencies are listed in requirements_libero.txt and requirements_robotwin.txt.

Download Checkpoints

Download the RealtimeWAM checkpoints from lightx2v/RealtimeWAM on Hugging Face. The simplest option is to download the checkpoints with the action expert's LoRA adapters already merged.

huggingface-cli download lightx2v/RealtimeWAM \
  realtimewam_libero_fast.pt \
  realtimewam_libero_faster.pt \
  realtimewam_robotwin_fast.pt \
  realtimewam_robotwin_faster.pt \
  libero_dataset_stats.json \
  robotwin_dataset_stats.json \
  --local-dir ./checkpoints/RealtimeWAM

Inference

Run the following scripts for BF16 or FP8 inference with RealtimeWAM^* and RealtimeWAM^†.

# RealtimeWAM*, based on Fast-WAM
bash scripts/realtimewam/run_libero_fastwam_i2va.sh

# RealtimeWAM†, based on Faster-WAM
bash scripts/realtimewam/run_libero_fasterwam_i2va.sh

# RealtimeWAM*, FP8
bash scripts/realtimewam/run_libero_fastwam_i2va_fp8.sh

# RealtimeWAM†, FP8
bash scripts/realtimewam/run_libero_fasterwam_i2va_fp8.sh

We provide example inference inputs in assets/inputs/realtimewam/.

Latency

The measured latency is reported in the logs under RealtimeWAM End-to-End Latency (Excluding Text Encoding), in milliseconds. It includes VAE encoding and video/action inference, while excluding text encoding.

Inference latency comparisons are shown below. Speedups are relative to the original model on the same GPU. The BF16 scripts above can be used to reproduce the inference latencies reported in our paper. LightX2V achieves slightly lower latency than reported in the paper. Our FP8 inference scripts can further reduce latency.

Memory

LightX2V also provides memory optimizations, including text encoder offload and FP8 quantization, to reduce GPU memory usage. The table below shows memory usage with LIBERO inputs, sampled after inference with warmup and CUDA Graph enabled. Values are in GiB and exclude initialization peaks.

Enable text encoder offload with "t5_cpu_offload": true. To load an offline FP8 checkpoint, set "dit_quantized": true, "dit_quant_scheme": "fp8-sgl", and "dit_quantized_ckpt" to its .pt path. Weights are loaded directly in FP8. Activations are quantized dynamically during inference. Allocated memory is measured with torch.cuda.memory_allocated(). nvidia-smi usage also includes PyTorch's cached memory, CUDA contexts, and other GPU allocations.

Evaluation

[Optional] Prepare Benchmark Resources

Skip this step if the benchmark assets are already installed. Use the repository-pinned benchmark versions; training demonstration datasets are not required for evaluation.

• LIBERO: Task definitions and initial states are included in the submodule.
• LIBERO-Plus: Download and extract assets.zip from Sylvest/LIBERO-plus on Hugging Face into LIBERO-plus/libero/libero/assets/.
• RoboTwin: Run bash script/_download_assets.sh from the RoboTwin root directory to download and configure the simulation assets.

For detailed preparation instructions, refer to the original repositories: LIBERO, LIBERO-Plus, and RoboTwin.

Run Evaluation

Run the following commands from the LightX2V repository root. Set the Wan2.2 model path and the GPUs to use:

export WAN_MODEL_PATH=/path/to/Wan2.2-TI2V-5B
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7

The examples below evaluate RealtimeWAM^* with one-step action denoising. Use the downloaded checkpoint and its matching dataset statistics. CKPT_PATH, DATASET_STATS_PATH, and CONFIG_JSON select the policy weights, normalization statistics, and inference configuration; OUT specifies the result directory.

For LIBERO, evaluate 40 tasks across four suites, with 50 trials per task. The policy starts after 30 settling steps:

source .venvs/libero/bin/activate
CKPT_PATH=./checkpoints/RealtimeWAM/realtimewam_libero_fast.pt \
DATASET_STATS_PATH=./checkpoints/RealtimeWAM/libero_dataset_stats.json \
CONFIG_JSON=configs/realtimewam/libero_fastwam_i2va.json \
OUT=evaluate_results/libero/realtimewam_fast \
bash scripts/bench/robotics/run_libero.sh model=realtimewam seed=42 \
  EVALUATION.num_trials=50 EVALUATION.num_steps_wait=30 EVALUATION.replan_steps=10

For LIBERO-Plus, reuse the LIBERO checkpoint and statistics:

source .venvs/libero/bin/activate
CKPT_PATH=./checkpoints/RealtimeWAM/realtimewam_libero_fast.pt \
DATASET_STATS_PATH=./checkpoints/RealtimeWAM/libero_dataset_stats.json \
CONFIG_JSON=configs/realtimewam/libero_fastwam_i2va.json \
OUT=evaluate_results/libero_plus/realtimewam_fast \
bash scripts/bench/robotics/run_libero_plus.sh model=realtimewam seed=42 \
  EVALUATION.num_trials=1 EVALUATION.num_steps_wait=30 EVALUATION.replan_steps=10

For RoboTwin 2.0, evaluate 50 tasks under both clean and randomized conditions, with 100 episodes per task and condition using unseen instructions:

source .venvs/robotwin/bin/activate
CKPT_PATH=./checkpoints/RealtimeWAM/realtimewam_robotwin_fast.pt \
DATASET_STATS_PATH=./checkpoints/RealtimeWAM/robotwin_dataset_stats.json \
CONFIG_JSON=configs/realtimewam/robotwin_fastwam_i2va.json \
OUT=evaluate_results/robotwin/realtimewam_fast \
bash scripts/bench/robotics/run_robotwin.sh model=realtimewam seed=42 \
  EVALUATION.eval_num_episodes=100 EVALUATION.instruction_type=unseen EVALUATION.replan_steps=24

To evaluate RealtimeWAM^†, replace the checkpoint suffix _fast.pt with _faster.pt, select the corresponding configuration below, and use a separate output directory.

All configurations are under configs/realtimewam/ and use one action denoising step.

The evaluation scripts above can also be used to reproduce the success rates reported in our paper. Results below are success rates (%). NFE denotes the number of denoising steps for video and action generation.

✏️ Citation

If you find our toolkit or research paper useful or relevant to your research, please kindly cite our work.

@article{lv2026realtimewam,
  title={RealtimeWAM: One-Step Asynchronous World Action Models},
  author={Lv, Chengtao and Du, Jinyang and Feng, Shuyi and Yong, Yang and Gu, Shiqiao and Yang, Shunzi and Gong, Ruihao and Ren, Shen and Zhang, Tianwei and Wang, Wenya},
  journal={arXiv preprint arXiv:2610.06617},
  year={2026}
}

🙌 Acknowledgements

Our implementation is based on FastWAM and FasterWAM. We thank the authors for sharing their code.

⚖️ License

The code is licensed under the Apache License 2.0. Pretrained models and third-party assets are governed by their respective licenses and terms.