🕹️ PixelGen

A text-to-pixel-art model with 1,257 parameters. Zero of them were trained.

Every weight was typed in by hand. It still works.

_{Every sprite above: "<colour> <class>" → PixelGen → 16×16 PICO-8 sprite. No gradients were harmed.}

---

What this is

A tiny PyTorch nn.Module that turns a text prompt like "red potion" or "blue invader" into a 16×16 pixel-art sprite in the PICO-8 palette, upscaled to 128×128.

It's an experiment in weight-space programming, inspired by this post: instead of training a network, you write a program and compile it into weights. Linear layers become lookup tables, argmax becomes a decision, a 1×1 conv becomes a palette. The model is a real nn.Module — you can call .forward(), save a state_dict, load it back — but no optimizer ever touched it.

The fun part is that an image generator with the parameter count of a single small MLP layer produces consistently cute, correctly-coloured, correctly-shaped sprites, and you can read the entire "model" as source code.

Numbers (measured, not estimated)

Parameter breakdown

How it works

The forward pass is nine steps. Each one is a normal tensor op; together they're a program.

"red potion"
    │
    ▼
 1. bag of words      prompt → count vector over a 59-word vocab
 2. class WTA         Linear(59→11) → argmax → one-hot class          [potion]
 3. shape bank        one-hot selects a 16×8 half-template;
                      '?' cells flip on when seed-noise ≥ 0.5
 4. mirror            half + horizontal flip → 16×16 body            (always symmetric)
 5. outline           3×3 dilate(body) − body → outline pixels       (palette index 1)
 6. shading           rows 0–6 light, 7–10 mid, 11–15 dark
 7. ramp WTA          Linear(59→9) → argmax → colour ramp             [red]
 8. one-hot palette   every pixel → one-hot over 16 PICO-8 colours
 9. palette conv      Conv2d(16→3, 1×1) → RGB
    │
    ▼
 16×16 sprite → nearest-neighbour ×8 → 128×128 PNG

The "generative" part is step 3: templates have ? cells that are gated by per-seed noise, so the same prompt gives small variations across seeds:

_{"blue invader", seeds 0 through 7}

How to use

git clone https://huggingface.co/DJLougen/pixelgen && cd pixelgen
pip install torch numpy pillow            # that's the whole dependency list
python generate.py "red potion" --seed 0 --out out/red_potion.png
python generate.py "golden key" --seed 3 --scale 16

Or from Python:

from pixelgen.model import PixelGen

model = PixelGen()                        # weights are built in __init__, no file needed
img = model.sprite("purple slime", seed=4, scale=8)   # PIL.Image, 128×128
img.save("slime.png")

The pixelgen.pt checkpoint is included for completeness (pixelgen.weights.load(path)), but it's redundant — PixelGen() reconstructs identical weights from source every time.

Prompt vocabulary. Class words (and synonyms):

Colour words: red blue green yellow purple orange pink brown white, plus synonyms (crimson ruby → red, azure sapphire → blue, emerald lime → green, gold golden → yellow, violet amethyst → purple, amber → orange, rose → pink, wood wooden → brown, grey gray silver bone → white). Anything else is ignored. Fun side effect: "bone" is both a skull synonym and a white synonym, so "bone" alone gives a white skull.

Limitations

Being upfront, because this is the part that matters:

• It only knows 11 things. Prompt for a dragon and you get a potion (the default class). There is no generalisation — zero — because there is no learned representation. The "model" is a lookup table with a tokenizer in front of it.
• Every sprite is horizontally symmetric. The mirror step is what makes 16×8 templates cheap, and it's also why the key had to be drawn as a top-down key rather than a side-view one.
• Variation is cosmetic. Seeds only toggle the hand-marked ? cells (a few pixels per sprite). Two seeds of the same prompt are the same sprite with slightly different speckles.
• Shading is positional, not structural. Light/mid/dark bands are fixed rows, so a tall sprite and a short sprite get lit differently for no good reason.
• Multi-word prompts resolve by word count, not meaning. "sword shield" is a tie broken by index order. "not red" is red.
• Colour choice is global. One ramp per sprite; no two-tone sprites.
• It's PyTorch for the bit, not for the need. You could rewrite this in 100 lines of NumPy. Using nn.Linear and nn.Conv2d is the point of the exercise, not a requirement.
• Art quality is "programmer art". The templates were hand-drawn in a text editor by someone who is not a pixel artist. invader and key went through a few iterations; the rest are first drafts.

Why bother?

Because it's a nice demonstration that a lot of what we call "a model" is just structure, and structure can be written directly. A linear layer with the right weights is a dictionary. A conv with the right kernel is an edge detector. Winner-take-all is an if. The blog post argues this is a useful lens for understanding what trained networks are doing; this repo is the version where you skip the training and just write the weights.

Also it's cool that it worked.

Repo contents

pixelgen/
  model.py       PixelGen nn.Module — all weights set in __init__
  templates.py   11 half-sprite templates ('#' solid, '?' noise, '.' empty)
  palette.py     PICO-8 palette + 9 colour ramps
  tokenizer.py   vocab + synonyms
  weights.py     save/load helpers
build.py         writes weights/pixelgen.pt (deterministic)
generate.py      CLI
tests/           10 property tests (shape, palette membership, determinism, symmetry, …)
weights/pixelgen.pt
showcase.png     the grid at the top
seeds.png        seed variation strip

Tests: python -m pytest tests -q → 10 passed.

Citation

@misc{pixelgen2025,
  title  = {PixelGen: a hand-weighted pixel-art generator},
  author = {Lougen, D. J.},
  year   = {2025},
  url    = {https://huggingface.co/DJLougen/pixelgen},
  note   = {Weight-space programming experiment; no parameters were trained.}
}

Inspired by Weight-Space Programming.