PosterReward: Unlocking Accurate Evaluation for High-Quality Graphic Design Generation

Overview

PosterReward is a dedicated reward modeling framework for poster assessment. It builds a 70k poster preference dataset from multi-MLLM consensus and introduces specialized models for poster quality evaluation across five dimensions:

1. Foundational Visual Quality
2. AI Artifacts
3. Textual Accuracy
4. Prompt Fidelity
5. Aesthetic Value

Available Models

This repository contains three model variants, all built on Qwen3-VL-8B:

PosterReward (Full Pipeline)

The full PosterReward pipeline is a two-stage analysis -> scoring process:

1. PosterReward Analyser generates a detailed textual analysis across five quality dimensions.
2. PosterReward Scorer takes the analysis together with the image and outputs a scalar reward score.

PosterReward-Lite

A simplified variant that directly predicts a scalar reward from the image and prompt, without requiring a separate analysis step. Faster inference at the cost of slightly lower accuracy.

Benchmark Data

This repository also hosts the PosterRewardBench benchmark images:

Download and extract these archives into the poster_reward_bench/ directory of the code repository.

Quick Start

Environment Setup

git clone https://github.com/MeiGen-AI/PosterReward.git
cd PosterReward

cd swift && pip install -e . && cd ..
pip install msgspec "qwen_vl_utils>=0.0.14" torchvision diffusers pillow

PosterReward-Lite (Fast Pointwise Scoring)

from swift.llm import PtEngine, InferRequest

model_path = "path/to/PosterReward-Lite"  # or download from this repo
engine = PtEngine(model_path, max_batch_size=64, task_type='seq_cls', num_labels=1)

messages = [
    {"role": "user", "content": "<image>Your poster description prompt here."},
    {"role": "assistant", "content": ""}
]
request = InferRequest(messages=messages, images=["path/to/poster.png"])

resp_list = engine.infer([request])
score = resp_list[0].choices[0].message.content
print(f"Reward Score: {score}")

Full PosterReward (Two-Stage Pipeline)

# Edit model paths in inference_posterreward.sh, then:
bash inference_posterreward.sh

Results

Pointwise Reward Models on PosterRewardBench

Citation

Coming Soon!

Acknowledgments

• Thanks to our collaborators and affiliated institutions.
• Thanks to the open-source community and prior reward modeling research.