RynnVLA-Latent-2B

Stage-1 latent-action pretraining checkpoint of RynnVLA-Latent, a Mixture-of-Transformers vision-language-action model. Given a language instruction and up to six camera views, it generates a chunk of latent actions by flow matching.

• Parameters: 2,510,917,041 (2.511 B), 1,186 tensors, all bfloat16
• Weights: EMA snapshot at the final training step 236,442
• Frozen during Stage 1: the vision tower and the language model (parameter regexes visual\., language_model\.)
• Text backbone: hidden 2048, 28 layers, 16 attention heads / 8 KV heads, intermediate 6,144
• Exported: 2026-09-28

Code, data format and training scripts: github.com/alibaba-damo-academy/RynnVLA-Latent

---

Read this first: what this checkpoint is not

This is a Stage-1 checkpoint. Its flow-matching target is the latent action, not a robot action, so Stage 1 consumes no action labels at all.

As shipped, processor_config.json carries an empty action/state schema ("schema": {"action": {}, "state": {}}, "use_state": false), and the eval adapters in the repository explicitly refuse a Stage-1 checkpoint for closed-loop control — validate_libero_schema raises "Use a Stage2 action checkpoint, not a Stage1 latent-only checkpoint" whenever use_latent_actions is true.

So: do not point LIBERO / RoboTwin / VLABench evaluation at this repo. Use it as the initialization for Stage 2, which keeps the same flow-matching objective and retargets it from latent actions to a robot-specific action space. action_dim = 81 and action_chunk_size = 30 are declared in the config and carried in the architecture, but Stage-1 training does not use them.

Latent-action protocol (608-dim ktoken_zcam)

Stage 1 trains against latent actions produced by RynnLAM, which labels raw RGB frame pairs and needs no action space — that is what lets real-robot trajectories, simulation rollouts and human egocentric video share one target representation.

Architecture

Up to 6 camera roles are carried per sample with a slot_mask marking the valid ones, so a source with one camera and a source with four train in the same batch.

Images are normalized with image_mean = image_std = [0.5, 0.5, 0.5], i.e. rescaled to [-1, 1] (processor_class = RynnBrainVLAProcessor). The separate action_norm_type field is mean_std; it governs action/state normalization and is inert at Stage 1, where schema.action and schema.state are both empty.

Loading

The custom model_type is resolved by the rynnvla package, so import it first:

import torch
from rynnvla.models.rynn_brain_vla import RynnBrainVLAModel, RynnBrainVLAProcessor

path = "./RynnVLA-Latent-2B"   # a local directory, e.g. from snapshot_download below
model = RynnBrainVLAModel.from_pretrained(
    path, dtype=torch.bfloat16, attn_implementation="flash_attention_2",
)
processor = RynnBrainVLAProcessor.from_pretrained(path)

Verified on transformers==5.2.0 / torch==2.7.1 with both attn_implementation="eager" and "flash_attention_2": the checkpoint materializes with zero missing and zero unexpected keys, the parameter count equals the safetensors header count exactly (2,510,917,041), and no tensor contains a non-finite value. The only exactly-zero tensors are the three action-head biases (state_proj.bias, action_in_proj.bias, action_out_proj.bias), which Stage 1 does not train; their weights are non-zero.

Download from ModelScope:

from modelscope import snapshot_download
model_dir = snapshot_download("DAMO_Academy/RynnVLA-Latent-2B")

Files

export_provenance.json records the source step, the EMA selection, the frozen-parameter regexes, the source weight digest and the tensor count. Internal absolute paths were removed from it before release, which changes only that file's own digest.

Training

Stage-1 latent-action pretraining on the full latent-action corpus, global batch size 512, run to completion at step 236,442. The released weights are the EMA snapshot of that final step.

License

Apache License 2.0.

Links

• Code & training scripts: github.com/alibaba-damo-academy/RynnVLA-Latent
• RynnLAM (produces the 608-dim latent actions): modelscope.cn/models/DAMO_Academy/RynnLAM
• Sibling checkpoints: RynnVLA-Latent-2B · RynnVLA-Latent-4B