mlx-community/flux-3-image-klein-pup-8bit
This is an MLX copy, 8-bit (affine, group 64), of black-forest-labs/flux-3-image-klein-pup (Apache-2.0), for Apple Silicon.
- Files: converted with mlx-vlm 0.7.8 (
mlx_vlm convert -q --q-bits 8 --q-group-size 64); tokenizer, chat template, processor configs andsystem_prompts.jsonare upstream's, byte-identical. - Other tiers: bf16, 4bit.
- Size: 9.8 GB.
The model is the prompt upsampler for FLUX 3 Image Klein. It's a Qwen3.5-9B vision-language fine-tune that expands a short request (plus up to 10 reference images, plus optional bounding boxes) into the full scene caption FLUX 3 Klein needs. A short prompt sent to the image model as typed gives generic output. This repo is Apache-2.0 like its upstream. The FLUX 3 Klein image model itself is under the FLUX Kommunity License (mlx-community/flux-3-image-klein-bf16).
Use
It loads with MLXVLM (mlx-swift-lm, qwen3_5) and with Python mlx-vlm. The upstream card is the reference for:
- the system prompts (
system_prompts.json:t2i,i2i,t2i_bbox,i2i_bbox); - the edit scaffold;
- the decoding: greedy, presence penalty 1.5 for text-to-image and 0 for edits,
enable_thinking=False, at most 3,584 new tokens.
Two details matter:
- The presence penalty counts generated tokens only, as vLLM does. mlx-lm's presence penalty also counts the prompt.
- References are prepared first: alpha over white, LANCZOS to 1 MP for one reference, 724² for two, 512² for 3–10, sides multiples of 32, then a JPEG q92 round trip.
flux3-klein-swift's Flux3PromptUpsampler implements all of it:
import Flux3KleinUpsampler
let up = try await Flux3PromptUpsampler.load(directory: snapshotURL)
let caption = try await up.upsample("A man walking with his dog, afternoon sun")
Parity (the Swift/MLX port vs BFL's reference, flux3/pup.py)
| Check | Result |
|---|---|
| Prompt tokens vs the HF processor (t2i, t2i+boxes, 1-ref and 2-ref edits, edit+boxes) | exact, 737–1,914 tokens |
Box planning / reconcile vs bbox.py |
13/13 cases exact |
| 1-ref edit caption on BFL's own reference pixels (bf16) | identical to BFL's caption, word for word |
| Text-to-image caption, man_dog (bf16) | 0.991 word similarity to BFL's |
Greedy decoding is chaotic near ties. BFL's own captions for the same request differ between their runs (0.25–1.0 similarity), so exact match is not the bar.
Tiers (19 requests from BFL's examples)
The bar: every quoted string kept verbatim and every box table complete. Similarity to the bf16 caption is reported for information.
| Tier | Quoted text kept | Box tables | Similarity to bf16 | Memory (M5 Max) |
|---|---|---|---|---|
| bf16 | 20/20 | 2/2 | 1.00 | 18.8 GB resident, 20.2 GB peak |
| 8bit | 20/20 | 2/2 | 0.68 | 10.4 GB resident, 11.1 GB peak |
| 4bit | 20/20 | 2/2 | 0.45 | 6.0 GB resident, 6.8 GB peak |
Lower tiers paraphrase more, but the captions keep the same structure, subjects and quoted text. Each caption takes about 9–15 s on an M5 Max.
License
Apache-2.0, as upstream. It is derived from Qwen/Qwen3.5-9B (Apache-2.0). This copy is not an official Black Forest Labs release.