ParallelTubeDecoding-Qwen3-VL-4B

This repository contains the merged Qwen3-VL-4B checkpoint for Parallel Tube Decoding (PTD) from Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding.

PTD first predicts the temporal interval of a queried event and then generates all time-conditioned spatial blocks in parallel. Decoupled Block Attention allows every spatial block to access the shared video-query context and the predicted temporal block while preventing dependencies between spatial blocks. The complete spatial tube is therefore produced in one parallel decoding round after temporal localization.

Code: mbzuai-oryx/ParallelTubeDecoding
Project page: Parallel Tube Decoding
Base model: Qwen/Qwen3-VL-4B-Instruct

Model details

Backbone: Qwen3-VL-4B-Instruct
Method: Parallel Tube Decoding (PTD)
Tasks: spatio-temporal video grounding and temporal localization
Spatial representation: 1,001 discrete coordinate tokens over [0, 1000]
Temporal representation: 100 discrete time tokens
PTD spatial block size: 6
Checkpoint format: merged model weights in FP16
Transformers version: 5.12.1

The checkpoint supports the released PTD inference path and Quantized (NTP) inference. It does not include data or a separate LoRA adapter.

Important loading note

Use this checkpoint with the released ParallelTubeDecoding code. Standard Transformers loading provides the Qwen3-VL architecture, but PTD generation and Decoupled Block Attention are implemented by the released codebase.

git clone https://github.com/mbzuai-oryx/ParallelTubeDecoding.git
cd ParallelTubeDecoding

conda create -n ptd python=3.11 -y
conda activate ptd
pip install --upgrade pip
pip install -r requirements.txt

The release uses:

transformers==5.12.1

For evaluation, set the model path to this Hugging Face repository:

export MODEL="MBZUAI/ParallelTubeDecoding-Qwen3-VL-4B"

Then follow the repository's evaluation instructions. The released adapter is based on lmms-eval v0.7.1 at commit 88b23e2bfa16a1edbc16e9e238ed82130b3a4f56.

Evaluation configuration

The paper evaluation uses:

fps=2
max_num_frames=64
min_pixels=131072
max_pixels=786432
temporal_patch_size=1

For PTD inference, use:

attn_implementation=sdpa
vision_attn_implementation=flash_attention_2
ptd_attn_implementation=flash_attention_2

Quantized (NTP) inference uses:

attn_implementation=sdpa

FlashAttention-2 must match the installed PyTorch and CUDA/ROCm environment.

Training

The released model was trained in two stages:

Supervised fine-tuning with the joint NTP/MTP formulation on the training splits of VidSTG and HC-STVG v1/v2.
Localization-aware GRPO with temporal IoU and spatial GIoU/L1 rewards.

Both stages used LoRA with rank 32 and alpha 64 while optimizing the newly introduced localization-token embeddings. The released checkpoint contains the merged weights. Charades-STA and ActivityNet Captions were not used for SFT; results on both are zero-shot.

The training video configuration was:

video_max_pixels=151200
fps=2
max_frames=64
temporal_patch_size=1

No training data, GRPO data, or GRPO data-selection code is included with the model release.

Results

PTD is evaluated on VidSTG, HC-STVG v1/v2, Charades-STA, and ActivityNet Captions. On the decoding-efficiency protocol, PTD achieves a Tube Completion Latency (TCL) of 0.4 seconds and 45.9 Boxes Per Second (BPS). Relative to unquantized autoregressive decoding, this corresponds to 79x lower TCL and 92x higher BPS while improving grounding accuracy.

The efficiency measurements use BF16, batch size 1, and a single 64-GB AMD Instinct MI210 GPU with synchronized decode-only timing. Refer to the paper and project page for complete benchmark tables and comparisons.

Intended use

This model is intended for research on:

spatio-temporal grounding of a referred entity in video;
temporal localization of described events;
efficient structured localization with PTD; and
evaluation on VidSTG, HC-STVG, Charades-STA, and ActivityNet Captions.

Users are responsible for complying with the licenses and terms of the source videos and evaluation datasets.

Limitations

The released formulation predicts one continuous temporal interval and one spatial tube per query. It is not designed for multiple disjoint event occurrences or multiple simultaneously valid instances. Temporally subtle state changes can produce ambiguous boundaries, while small, fast-moving, or occluded targets remain challenging.

Citation

@misc{rasheed2026locateanythingvideos,
  title  = {Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding},
  author = {Hanoona Rasheed and Haania Siddiqui and Ming-Hsuan Yang and Fahad Shahbaz Khan and Salman Khan},
  year   = {2026}
}

Acknowledgements

This work builds on Qwen3-VL and the Qwen-VL-Series-Finetune training framework. See the code repository for full acknowledgements.

MBZUAI/ParallelTubeDecoding-Qwen3-VL-4B