ProactiveMobile Agent (Qwen2.5-VL-7B-Instruct)

A proactive mobile-assistant model fine-tuned from Qwen2.5-VL-7B-Instruct on the ProactiveMobile benchmark. Given a user's context (profile, phone state, world knowledge, and an operation trace that is either GUI screenshots or a textual action sequence), the model decides whether to proactively act and, if so, emits a structured sequence of function calls — or correctly stays silent (no-action) when nothing should be recommended.

• Base model: Qwen/Qwen2.5-VL-7B-Instruct
• Task: proactive recommendation via function calling, over TEXT and GUI contexts
• Dataset: 🤗 xiaomi-research/ProactiveMobile
• Code: github.com/xiaomi-research/proactive-mobile

Output format

The model produces a three-part output; only the <function> block is scored:

<think>reasoning</think><rec>natural-language recommendation</rec><function>{"model_recommendation": [...]}</function>

Usage

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "<your-hf-id>/ProactiveMobile-Qwen2.5-VL-7B",
    torch_dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("<your-hf-id>/ProactiveMobile-Qwen2.5-VL-7B")

For batched benchmark inference and scoring (Type-Acc / Precision / Recall / F1 / FTR), see evaluation/inference_and_eval.py in the code repository.

Evaluation

Evaluated on the full ProactiveMobile benchmark (3,658 instances) with greedy decoding (temperature = 0), which gives deterministic, fully reproducible results.

All scores are percentages.

• Type-Acc: function-name sequence exact match against the best-matching gold intent.
• Precision / Recall / F1: set overlap of predicted vs. gold function names.
• FTR (False Trigger Rate, lower is better): on no-action instances, the fraction where the model nonetheless predicted an action.

License

This model is a derivative of Qwen2.5-VL-7B-Instruct (Apache 2.0), so the model weights are distributed under Apache 2.0. It was fine-tuned on the ProactiveMobile dataset, which is released under CC BY-NC-SA 4.0; use of this model is therefore subject to that dataset license — non-commercial use only, attribution required, share-alike.

Citation

@inproceedings{kong2026proactivemobile,
  title     = {ProactiveMobile: A Comprehensive Benchmark for Boosting Proactive Intelligence on Mobile Devices},
  author    = {Kong, Dezhi and Feng, Zhengzhao and Liang, Qiliang and others},
  booktitle = {CVPR},
  year      = {2026}
}