RynnVLA-Latent-4B

Stage-1 latent-action pretraining checkpoint of RynnVLA-Latent, a Mixture-of-Transformers vision-language-action model. Given a language instruction and up to six camera views, it generates a chunk of latent actions by flow matching.

• Parameters: 5,042,270,129 (5.042 B), 1,426 tensors, all bfloat16
• Weights: EMA snapshot at the final training step 236,442
• Frozen during Stage 1: the vision tower and the language model (parameter regexes visual\., language_model\.)
• Text backbone: hidden 2560, 36 layers, 32 attention heads / 8 KV heads, intermediate 9,728
• Exported: 2026-10-03

Code, data format and training scripts: github.com/alibaba-damo-academy/RynnVLA-Latent

---

Read this first: what this checkpoint is not

This is a Stage-1 checkpoint. Its flow-matching target is the latent action, not a robot action, so Stage 1 consumes no action labels at all.

As shipped, processor_config.json carries an empty action/state schema ("schema": {"action": {}, "state": {}}, "use_state": false), and the eval adapters in the repository explicitly refuse a Stage-1 checkpoint for closed-loop control — validate_libero_schema raises "Use a Stage2 action checkpoint, not a Stage1 latent-only checkpoint" whenever use_latent_actions is true.

So: do not point LIBERO / RoboTwin / VLABench evaluation at this repo. Use it as the initialization for Stage 2, which keeps the same flow-matching objective and retargets it from latent actions to a robot-specific action space. action_dim = 81 and action_chunk_size = 30 are declared in the config and carried in the architecture, but Stage-1 training does not use them.

Latent-action protocol (608-dim ktoken_zcam)

Stage 1 trains against latent actions produced by RynnLAM, which labels raw RGB frame pairs and needs no action space — that is what lets real-robot trajectories, simulation rollouts and human egocentric video share one target representation.

Architecture

Up to 6 camera roles are carried per sample with a slot_mask marking the valid ones, so a source with one camera and a source with four train in the same batch.

Images are normalized with image_mean = image_std = [0.5, 0.5, 0.5], i.e. rescaled to [-1, 1] (processor_class = RynnBrainVLAProcessor). The separate action_norm_type field is mean_std; it governs action/state normalization and is inert at Stage 1, where schema.action and schema.state are both empty.

Loading

The custom model_type is resolved by the rynnvla package, so import it first:

import torch
from rynnvla.models.rynn_brain_vla import RynnBrainVLAModel, RynnBrainVLAProcessor

path = "./RynnVLA-Latent-4B"   # a local directory, e.g. from snapshot_download below
model = RynnBrainVLAModel.from_pretrained(
    path, dtype=torch.bfloat16, attn_implementation="flash_attention_2",
)
processor = RynnBrainVLAProcessor.from_pretrained(path)

Verified on transformers==5.2.0 / torch==2.7.1 with both attn_implementation="eager" and "flash_attention_2": the checkpoint materializes with zero missing and zero unexpected keys, the parameter count equals the safetensors header count exactly (5,042,270,129), and no tensor contains a non-finite value. The only exactly-zero tensors are the three action-head biases (state_proj.bias, action_in_proj.bias, action_out_proj.bias), which Stage 1 does not train; their weights are non-zero.

Download from ModelScope:

from modelscope import snapshot_download
model_dir = snapshot_download("DAMO_Academy/RynnVLA-Latent-4B")

Files

export_provenance.json records the source step, the EMA selection, the frozen-parameter regexes, the source weight digest and the tensor count, plus an export_file_sha256 manifest over the six sibling files. Internal absolute paths were removed from it before release, which changes only that file's own digest; the sibling manifest deliberately does not cover it, so all six recorded digests still match.

Training

Stage-1 latent-action pretraining on the full latent-action corpus, global batch size 512, run to completion at step 236,442. The released weights are the EMA snapshot of that final step, and the frozen trunk digest is recorded as e577269c42fd307c….

License

Apache License 2.0.

Links

• Code & training scripts: github.com/alibaba-damo-academy/RynnVLA-Latent
• RynnLAM (produces the 608-dim latent actions): modelscope.cn/models/DAMO_Academy/RynnLAM
• Sibling checkpoints: RynnVLA-Latent-2B · RynnVLA-Latent-4B