Qwen3-VL-8B-Instruct-MLX-4bit

MLX conversion of Qwen/Qwen3-VL-8B-Instruct for Apple Silicon.

• Converted from the base model at revision 0c351dd01ed87e9c1b53cbc748cba10e6187ff3b.
• Not affiliated with or endorsed by Qwen.
• Converted by Sasan Sotoodehfar, CAVI AI (https://cavi-ai.xyz).

Contents

• model-00001-of-00002.safetensors, model-00002-of-00002.safetensors: 5.76 GB (5.37 GiB) in total.
• Quantized to 4-bit (affine, group size 64): the language model's linear layers, token embedding and output head (36 layers, hidden size 4096).
• Kept in bf16: the vision tower (27 blocks) and its merger, and the language model's norms.
• Processor, tokenizer and chat template files from the base model.

Requirements

• Apple Silicon Mac.
• Python 3.10 or later.
• pip install mlx-vlm==0.6.10

Usage

from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("cavi-ai/Qwen3-VL-8B-Instruct-MLX-4bit")
prompt = apply_chat_template(
    processor, model.config,
    'Where is the Add customer button? Answer with its center as JSON {"x": <0-1000>, "y": <0-1000>}.',
    num_images=1,
)
print(generate(model, processor, prompt, ["screenshot.png"], max_tokens=40).text)

• Qwen3-VL gives points on a 0-1000 grid over the image: multiply x by width / 1000 and y by height / 1000 for pixels.
• On a 1900×980 web page screenshot the example answers {"x": 953, "y": 95}, the center of the button (1811, 93 in pixels).

Measured results

Hardware: Apple M5 Max.

• Screenshots are 1900×980. The first request after loading takes 7.0 s.
• Measured through the bobby-browser vision server with its mlx-vlm provider, on bobby-browser's hand-built test pages.

License

• Model weights, tokenizer, and configuration files: Apache-2.0, inherited from the base model. See LICENSE.

Links

• Base model: https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct
• Converted with mlx-agent: https://github.com/cavi-ai/mlx-agent