MOSS-Image-SFT for Apple Silicon (MLX, compact)
An MLX build of OpenMOSS-Team/MOSS-Image-SFT, OpenMOSS's 5B text-to-image DiT for long bilingual text rendering and JSON layout control, for Macs with Apple Silicon. Runs with moss-image-mlx, a port verified against the official PyTorch pipeline.
- 7.42 GB peak memory for a 1024² image with a short prompt (about +1 GB with a 650-token prompt), against a 27 GB download and a 16.6 GB peak for the original bf16 weights.
- all eval text still rendered correctly (14-image evaluation below).
- Official seeds reproduce official images.
- Download: 12.1 GB. Transformer 8-bit; text encoder (Qwen3.5-9B language path) 6-bit; VAE decoder fp32.
Renders from this tier at 1024², 50 steps, CFG 3, seed 42. More in assets/.
Quickstart
pip install moss-image-mlx
moss-mlx "A cozy bakery at dusk. A wooden sign reads \"THE GOLDEN CRUST BAKERY\"." --model VisualInference/MOSS-Image-SFT-MLX-compact -o bakery.png
from moss_image_mlx import MOSSImagePipeline, Request, save
from huggingface_hub import snapshot_download
pipe = MOSSImagePipeline(snapshot_download("VisualInference/MOSS-Image-SFT-MLX-compact"))
req = Request(prompt="A neon sign that reads \"OPEN LATE\" above a ramen bar", width=1024, height=1024, seed=7)
save(pipe(req), req, "ramen.png")
Layout prompts (JSON), batch rendering, the prompt cache and every option are documented in the GitHub README.
Which tier?
| Tier | Download | Peak at 1024² | Pick it if |
|---|---|---|---|
| 8bit | 14.1 GB | 9.3 GB | you have 24 GB or more; reference quality |
| compact | 12.1 GB | 7.4 GB | you have 16 GB |
original bf16 (--model OpenMOSS-Team/MOSS-Image-SFT) |
27.2 GB | 16.6 GB | you have 32 GB or more; fastest (~11.5 min per image), no quantization |
Measured on a Mac mini M4 Pro (48 GB), MLX 0.32.4, one 1024² image at the official defaults (50 steps, CFG 3, shift 5, block 22 skipped in the negative branch). Time: 12.4 min per image. The model is compute-bound, so quantization saves memory, not time.
Quality evidence
The same 14 prompts were rendered with the bf16 weights and with this tier, then scored:
- OCR: Apple Vision, order-free character recall against the requested text.
- Layout: IoU between each requested text box and where the text actually appears.
| Prompt (1024², seed 42) | bf16 reference | This tier |
|---|---|---|
| Bakery sign + chalkboard (EN): OCR recall | 0.98 | 0.94 |
| Book cover, 3 lines (EN): OCR recall | 1.00 | 1.00 |
| Typed 4-line paragraph (EN): OCR recall | 1.00 | 1.00 |
| 3-step infographic (EN): OCR recall | 0.99 | 0.99 |
| Handwritten fridge note (ZH): OCR recall | 1.00 | 0.96 |
| Tea-house calligraphy sign (ZH) ¹: OCR recall | 0.30 | 0.40 |
| Event poster, 3 lines (ZH): OCR recall | 1.00 | 1.00 |
| Printed notice, 4 lines (ZH): OCR recall | 1.00 | 1.00 |
| Layout JSON: EXPLORE poster: box IoU per text | 0.96 | 0.98 |
| Layout JSON: jazz poster, 3 text boxes: box IoU per text | 0.88, 0.93, 0.66 | 0.86, 0.93, 0.80 |
Summary:
- OCR mean 0.910 (bf16: 0.908).
- Layout IoU 0.892 (bf16: 0.858).
- Mean PSNR vs the bf16 render 27.5 dB. With this model, bf16 rounding alone moves an image about 23 dB.
¹ Apple's OCR cannot read the vertical calligraphy in this prompt for any variant, bf16 included; the render itself is correct.
The remaining misses were checked by eye and are OCR misreads of correct renders (for example 顺 read as 順). More: how the port was verified.
What is quantized:
- Transformer: the 42 transformer blocks' linear layers use MLX affine 8-bit weights (group size 64). The patch embedder, timestep MLP, shared modulation, head modulation, final projection and text projection stay bf16 (~1% of weights).
- Text encoder: linear layers and embeddings use 6-bit, group size 64. Its RMSNorm weights are kept in fp32 to match the reference.
- VAE: the decoder only, fp32.
Limitations
- Dense, very small text can garble, as in the original model.
- The SFT checkpoint shows faint vertical banding on some skies and walls (upstream behaviour).
- ~12.4 min per 1024² image on an M4 Pro at the official 50 steps.
--turbo(moss-image-mlx ≥ 0.2.0) makes a ~1 MP image in ~2 min with a trained student: MOSS-Image-SFT-Turbo-MLX. It was measured on the 8bit tier;--turboon this compact tier is untested.
中文说明
本仓库是 OpenMOSS MOSS-Image-SFT 的 Apple Silicon（MLX）量化版本，配合 moss-image-mlx 使用。
- 显存与下载： 生成一张 1024² 图像的峰值内存约 7.42 GB，下载 12.1 GB。
- 正确性： 已与官方 PyTorch 流程逐项对齐（采样日程逐位一致，fp32 端到端 69.7 dB）。
- 种子复现： 使用官方种子可复现官方示例图。
- 文字渲染： 在 14 张图的评测中，中英文文字渲染与 bf16 原版均正确。
安装：pip install moss-image-mlx，然后运行 moss-mlx "提示词" --model VisualInference/MOSS-Image-SFT-MLX-compact。
License and attribution
Apache-2.0, the same as the original weights. See NOTICE.
- MOSS-Image: © OpenMOSS.
- Text encoder: Qwen3.5-9B (Apache-2.0).
- VAE: FLUX.2 with the FLUX.2 small decoder by Black Forest Labs (Apache-2.0).
This is an independent conversion, not affiliated with or endorsed by OpenMOSS.