MOSS-VL-Base-0708

MOSS-VL is an open vision-language model family from OpenMOSS, supporting image understanding, long-video understanding, and realtime streaming interaction.

Technical Report: https://arxiv.org/pdf/2608.15045

Introduction

MOSS-VL-Base-0708 is the foundation checkpoint of the MOSS-VL 0708 release, part of the OpenMOSS ecosystem for open visual understanding.

Built through multimodal pretraining only, this checkpoint serves as a high-capacity offline multimodal base model. It provides strong general-purpose visual-language representations across image and video inputs, and is intended primarily as the base model for supervised fine-tuning, alignment, and domain adaptation.

The 0708 release keeps the MOSS-VL cross-attention design and a 256K text context window while refreshing the data and pretraining recipe for stronger offline multimodal foundations.

Specifically, the pretraining pipeline follows four progressive stages:

Stage 1: Vision-language alignment
Stage 2: Large-scale multimodal pretraining
Stage 3: High-quality multimodal pretraining
Stage 4: Annealing and long-context extension

Highlights

Strong foundation model: provides general visual-language representations for image, video, and text inputs.
Native dynamic resolution: processes images and video frames at their original aspect ratios and resolutions.
Native interleaved image and video inputs: supports mixed image/video/text sequences in a unified pipeline.
Open base checkpoint: designed for continued pretraining, supervised fine-tuning, alignment, and domain adaptation.

Model Architecture

MOSS-VL-Base-0708 adopts a cross-attention-based vision-language architecture that decouples visual encoding from language reasoning. The model processes images, videos, and text in a unified pipeline and uses cross-attention layers to connect language tokens with visual representations.

Key configuration details:

| Item | Value |

| --- | --- |

| Parameters | 11B |

| Tensor type | BF16 |

| Context length | 256K |

| Vision patch size | 16 |

| Temporal patch size | 1 |

| Default video FPS | 1.0 |

| Default max video frames | 256 |

Absolute Timestamps

For video inputs, MOSS-VL injects absolute timestamps alongside sampled frames. This helps the base model learn event order, duration, pacing, and temporal localization instead of relying only on frame order.

Cross-attention RoPE (XRoPE)

MOSS-VL uses Cross-attention Rotary Position Embedding (XRoPE), which maps text tokens and visual patches into a unified three-dimensional coordinate space defined by Time (t), Height (h), and Width (w). This gives the model a consistent positional representation for image and video understanding.

Model Performance

MOSS-VL-Base-0708 is intended as a pretrained foundation checkpoint for offline multimodal understanding and model adaptation. Detailed benchmark tables for the 0708 release will be maintained in the MOSS-VL project resources.

For the previous public base checkpoint, see MOSS-VL-Base-0408.

Quickstart

Installation

Clone the MOSS-VL repository and install the project requirements:

git clone https://github.com/OpenMOSS/MOSS-VL.git
cd MOSS-VL
conda create -n moss_vl python=3.12 pip -y
conda activate moss_vl
pip install -i https://pypi.org/simple --no-build-isolation -r requirements.txt

Load Model

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

checkpoint = "OpenMOSS-Team/MOSS-VL-Base-0708"

processor = AutoProcessor.from_pretrained(
    checkpoint,
    trust_remote_code=True,
    frame_extract_num_threads=1,
)
model = AutoModelForCausalLM.from_pretrained(
    checkpoint,
    trust_remote_code=True,
    device_map="auto",
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
)

Run Inference

Single-image Inference

image_path = "data/example_image.jpg"

text = model.offline_image_generate(
    processor,
    prompt="",
    image=image_path,
    shortest_edge=4096,
    longest_edge=16777216,
    multi_image_max_pixels=201326592,
    patch_size=16,
    temporal_patch_size=1,
    merge_size=2,
    image_mean=[0.5, 0.5, 0.5],
    image_std=[0.5, 0.5, 0.5],
    max_new_tokens=256,
    temperature=1.0,
    top_k=50,
    top_p=1.0,
    repetition_penalty=1.0,
    do_sample=False,
    vision_chunked_length=64,
)

print(text)

Single-video Inference

video_path = "data/example_video.mp4"

text = model.offline_video_generate(
    processor,
    prompt="",
    video=video_path,
    shortest_edge=4096,
    longest_edge=16777216,
    video_max_pixels=201326592,
    patch_size=16,
    temporal_patch_size=1,
    merge_size=2,
    video_fps=1.0,
    min_frames=1,
    max_frames=256,
    num_extract_threads=4,
    image_mean=[0.5, 0.5, 0.5],
    image_std=[0.5, 0.5, 0.5],
    max_new_tokens=256,
    temperature=1.0,
    top_k=50,
    top_p=1.0,
    repetition_penalty=1.0,
    do_sample=False,
    vision_chunked_length=64,
)

print(text)

Batched Offline Inference

offline_batch_generate accepts independent image/video/text queries. Queries in the same batch should share the same media_kwargs and generate_kwargs.

queries = [
    {
        "images": ["data/sample_a.jpg"],
        "videos": [],
        "generate_kwargs": {
            "temperature": 1.0,
            "top_k": 50,
            "top_p": 1.0,
            "max_new_tokens": 256,
            "repetition_penalty": 1.0,
            "do_sample": False,
        },
    },
    {
        "images": [],
        "videos": ["data/sample_b.mp4"],
        "media_kwargs": {
            "video_fps": 1.0,
            "min_frames": 8,
            "max_frames": 256,
        },
        "generate_kwargs": {
            "temperature": 1.0,
            "top_k": 50,
            "top_p": 1.0,
            "max_new_tokens": 256,
            "repetition_penalty": 1.0,
            "do_sample": False,
        },
    },
]

with torch.no_grad():
    result = model.offline_batch_generate(
        processor,
        queries,
        vision_chunked_length=64,
    )

texts = [item["text"] for item in result["results"]]
print(texts)

Related Checkpoints

| --- | ---: | ---: | --- | --- |

Limitations and Future Work

MOSS-VL-Base-0708 is a pretrained base checkpoint. It is not instruction-tuned, so applied use cases should generally fine-tune or align it before using it as an assistant-style model.

We are continuing to improve OCR and document understanding, extremely long video understanding, mathematical reasoning, code reasoning, RL post-training, and broader task-specific evaluations for future MOSS-VL releases.

Citation

@misc{mossvl,
  title         = {MOSS-VL Technical Report},
  author        = {Wang, Pengyu and Tan, Chenkun and Zhou, Shaojun and Zhou, Qirui and Chen, Yanxin and He, Xingyang and Zeng, Huazheng and Cheng, Jijun and Wang, Chenghao and Qian, Xiaomeng and Wang, Pengfei and Huang, Zhan and Gao, Shanqing and Huang, Wei and Cao, Longjun and Ran, Wu and Liu, Jie and Zhu, Changtai and Wang, Hongkai and Tian, Yixian and Liu, Chenghao and Ye, Zhen and Wang, Xinghao and Jiang, Botian and Feng, Guoguo and Fei, Zhaoye and Li, Ruixiao and Chen, Mingshu and Gao, Yang and Cheng, Qinyuan and Li, Shimin and Qiu, Xipeng},
  year          = {2026},
  eprint        = {2608.15045},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2608.15045}
}

@misc{mossvideopreview,
  title         = {{MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention}},
  author        = {Pengyu Wang and Chenkun Tan and Shaojun Zhou and Wei Huang and Qirui Zhou and Zhan Huang and Zhen Ye and Jijun Cheng and Xiaomeng Qian and Yanxin Chen and Xingyang He and Huazheng Zeng and Chenghao Wang and Pengfei Wang and Hongkai Wang and Shanqing Gao and Yixian Tian and Chenghao Liu and Xinghao Wang and Botian Jiang and Xipeng Qiu},
  year          = {2026},
  eprint        = {2606.07639},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2606.07639}
}