ERNIE-Image-Turbo OpenVINO INT4 AWQ
An OpenVINO weight-only quantization of the 8B ERNIE-Image-Turbo transformer. The model keeps its BF16 Mistral3 text encoder and Flux2 VAE from the upstream Diffusers checkpoint, while running the transformer in OpenVINO. The optional prompt enhancer is omitted; prompts are sent directly to the native text encoder.
Quick start
Use Python 3.11 and install a PyTorch build that matches your system. Then install the remaining pinned dependencies:
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
Generate a 1024×1024 poster:
python infer.py \
--prompt "A polished coffee shop poster with a warm cream background. Include exactly: CEDAR & BEAN, OPEN DAILY, 7 AM - 8 PM. Elegant typography and a ceramic cup beside coffee beans." \
--output poster.png \
--steps 8 \
--seed 42
OpenVINO runs the transformer on CPU. Set --torch-device cuda to run the native text encoder and VAE on an available NVIDIA GPU; this is the validated and recommended route. CPU execution is available but substantially slower, especially for VAE decoding. The OpenVINO transformer supports dynamic image and text dimensions; image sizes must be divisible by 16. The prompt enhancer is not included, so use_pe is disabled.
Quantization
NNCF 3.4.0 applied INT4_SYM group-size-64 compression with ratio 0.9 and AWQ to eight captured transformer calls. Ten percent of ratio-defining parameters use per-channel INT8_SYM backup. The OpenVINO transformer is 4.65 GB, compared with a 30.0 GB uncompressed FP32 IR fallback. The text encoder and VAE remain BF16 safetensors from the upstream repository. quantization_config.json records the recipe.
Validation
The included sample was generated end to end with the quantized OpenVINO CPU transformer, BF16 text encoder and VAE on CUDA, 1024×1024 resolution, eight steps, guidance 1, and seed 42. On the 30-vCPU Lambda host, transformer-only OpenVINO inference plus native CUDA components took 368.6–402 seconds across two runs. The BF16 CUDA reference took 27.4 seconds on the same host; this is a hardware-path comparison, not a like-for-like speed claim. The quantized image preserved the requested poster text and structure, with normal seed-sensitive changes in image details.
Source and license
Base model: baidu/ERNIE-Image-Turbo. The upstream model is Apache-2.0; see LICENSE. The OpenVINO conversion is provided under the same license. For the upstream model's recommended parameters and license terms, see the source model card.