Tiny text-to-image models (1.9M–8.8M parameters) for 16×16 pixel-art sprites. They work directly in pixel space, with no VAE and no pretrained text encoder, and run in a browser with WebGPU.
M-P1, 16 steps, seeds 0–7. Captions top to bottom: red potion bottle · side-view knight with sword · green slime · golden key · wooden treasure chest · blue mage hat. Not cherry-picked.
Try it in your browser → (WebGPU, falls back to CPU; captions and images stay on your device; the page counts visits with cookie-free GoatCounter)
Code: github.com/AnticPan/mini-pixel-art · Weights: huggingface.co/AnticPan/mini-pixel-art
| Model | Params | Width × blocks | Patch | Web demo weights | Notes |
|---|---|---|---|---|---|
s-p1 |
1.93M | 128 × 6 | 1×1 | fp32, 7.7 MB | Smallest; noticeably lower subject accuracy |
m-p1 |
4.35M | 192 × 8 | 1×1 | int8, 4.5 MB | Recommended balance of quality, size and speed |
l-p1 |
8.75M | 256 × 10 | 1×1 | int8, 9.0 MB | Best quality; about 1.8× slower than m-p1 |
m-p2 |
4.36M | 192 × 8 | 2×2 | fp32, 17.4 MB | Each token covers 2×2 pixels (64 pixel tokens instead of 256): about 2.5× faster than m-p1, a bit less detail |
Left to right: s-p1, m-p1, l-p1, with the same noise (seeds 0–3) and the same six captions as above.
All models are single-stream DiTs with 4 heads. Text is split into words and looked up in a 10,944-word table built from the training captions. Each image is predicted as continuous RGBA in [-1, 1], then each pixel is snapped to the nearest colour in the dataset's 4,096-colour palette (pixels with alpha < 0 become transparent).
pip install torch numpy safetensors pillow
python mini_pixel_art.py --model m-p1 --n 8 "red potion bottle" "side-view knight with sword" --out sheet.png
from mini_pixel_art import Pipeline
pipe = Pipeline(".", "m-p1") # or device="cuda"
rgba = pipe(["golden key", "green slime"], seed=0) # uint8 [2, 16, 16, 4]
ONNX (no PyTorch): each model also ships as model.onnx (opset 17, fp32, dynamic batch). Inputs are z [B,4,16,16] float32, t [B] float32 (1 = noise, 0 = clean) and cap_ids [B,12] int64; the output x is the predicted clean image. mini_pixel_art_onnx.py contains the same sampling loop in numpy:
pip install numpy onnxruntime pillow
python mini_pixel_art_onnx.py --model m-p1 --n 8 "golden key" "green slime" --out sheet.png
On the same noise, the ONNX models match PyTorch to within 2e-5 per forward pass, and 16-step generation gives identical pixels. Numpy noise differs from torch noise, so the same --seed gives different images in the two scripts.
Default sampler (in each config.json): 16 Euler steps, classifier-free guidance 1.5 on RGB and 1.5 on alpha (1.0 on alpha for l-p1), and the grid stops at t = 0.05, with the last step jumping straight to the prediction. Keep t_stop = 0.05. The models were trained with max(t, 0.05) in the loss, so their predictions below t = 0.05 are biased. A uniform grid with more than 16 steps evaluates there and gets worse as you add steps.
Captions work best when written like the training data: short noun phrases, optionally starting with a view word (front-view, three-quarter-view, top-down, side-view). Words not in the vocabulary are ignored; the script prints them.
All numbers use 4,096 held-out captions with their real images. These are rows randomly held out from the official train split and never used in training. The official validation split has no view words and shorter captions, so it is only used to pick sampler settings.
| Model | Params | Subject hit ↑ (real 0.733) | Feature Fréchet ↓ | Real/fake AUC ↓ (0.5 = indistinguishable) | Colours / image (real 22.3) |
|---|---|---|---|---|---|
| s-p1 | 1.93M | 0.609 | 6.02 | 0.557 | 53.6 |
| m-p1 | 4.35M | 0.692 | 5.41 | 0.533 | 50.7 |
| l-p1 | 8.75M | 0.714 | 4.96 | 0.528 | 47.9 |
| m-p2 | 4.36M | 0.684 | 5.84 | 0.536 | 66.7 |
| UbaNova-16 | 42.4M | 0.448 | 9.62 | 0.786 | 21.1 |
| tessera-small | 47.6M | 0.693 | 13.31 | 0.703 | 19.5 |
| pixelsprite-canvas-v2 | 39.1M | 0.430 | 7.16 | 0.630 | 23.8 |
t_stop 0.05) with fp32 compute, and the same weights as the web demo: int8 for m-p1 and l-p1, fp32 for s-p1 and m-p2.Speed for 16 steps with guidance, excluding text preprocessing:
| s-p1 | m-p1 | l-p1 | m-p2 | |
|---|---|---|---|---|
| Browser, WebGPU (Apple-silicon Mac, batch 8) | ~70 ms / image | ~130 ms | ~210 ms | ~48 ms |
| PyTorch CPU, 4 threads, batch 1, fp32 | ~134 ms | ~250 ms | ~430 ms | ~100 ms |
| RTX 5090, batch 1024, bf16 | 1,799 images / s | 873 | 532 | 3,168 |
mini_pixel_art.py standalone inference (torch only)
mini_pixel_art_onnx.py the same with onnxruntime + numpy
<model>/model.safetensors fp32 EMA weights, for <model> in s-p1, m-p1, l-p1, m-p2
<model>/model.onnx the same weights as ONNX (opset 17, fp32)
<model>/config.json architecture and default sampler
vocab.json word -> id (ids 0/1/2 are pad / unk / null)
palette.bin 4097 × RGB uint8 (index 0 = transparent)
docs/index.html the web demo (served by GitHub Pages from /docs)
docs/models/ weights for the demo, downloaded only when a model is selected
Code and weights: MIT.
The weights were trained from scratch. They contain no part of any other model's weights. The only training data was Angais/pixel-art-atlas-1.4m, released under CC0 1.0. According to its card, its sprites were generated with Krea 2 Turbo and a pixel-art LoRA, and its CC0 dedication does not override the Krea 2 Community License or the Krea Acceptable Use Policy. See NOTICE. This project is not affiliated with or endorsed by Krea or the dataset author.
l-p1/model.onnx 35 MBl-p1/model.safetensors 35 MBm-p1/model.onnx 18 MBm-p2/model.onnx 18 MBm-p2/model.safetensors 17 MBm-p1/model.safetensors 17 MBs-p1/model.onnx 8 MBs-p1/model.safetensors 8 MBpalette.bin 12.3 kB