MOSS-Image PixelArt 16x16

A LoRA for MOSS-Image-Base that draws 16x16 pixel-art game icons: one item per image, dark outline, flat shading, on a flat magenta (#FF00FF) background that comes off with a simple colour key, including the holes inside handles, rings and loops. Apache-2.0, like the base model.

Uncurated outputs, background removed by colour key. Objects absent from the training set (snowman, anchor, chess knight, sushi, ice cream cone, space helmet, teapot) come out as well as the trained ones.

How to use

512x512, 50 steps, guidance 3 (MOSS-Image's recommended sampler settings). Prompt template:

pixel art 16x16 game icon, <your object and details>, dark outline, plain magenta background

MOSS-Image ships its pipeline as code inside the base repo, and diffusers has no LoRA loader for it yet, so the snippet builds the pipeline from that code and merges the LoRA into the transformer weights. Tested with diffusers 0.41.0 and transformers 5.19.0. Everything on the GPU takes about 30 GB; on smaller cards, pipe.enable_model_cpu_offload() instead of .to("cuda") should keep only the active part on the GPU.

import sys

import numpy as np
import torch
from diffusers import AutoencoderKLFlux2
from huggingface_hub import hf_hub_download, snapshot_download
from PIL import Image
from safetensors.torch import load_file
from transformers import AutoModel, AutoTokenizer

base = snapshot_download("OpenMOSS-Team/MOSS-Image-Base", ignore_patterns=["assets/*"])
sys.path.insert(0, base)
import pipeline_moss_image as pm

pipe = pm.MOSSImagePipeline(
    pm.MOSSImageTransformer.from_pretrained(f"{base}/transformer", torch_dtype=torch.bfloat16),
    pm.MOSSImageScheduler.from_pretrained(f"{base}/scheduler"),
    AutoModel.from_pretrained(f"{base}/text_encoder", dtype=torch.bfloat16),
    AutoTokenizer.from_pretrained(f"{base}/tokenizer"),
    AutoencoderKLFlux2.from_pretrained(f"{base}/vae", torch_dtype=torch.float32),
).to("cuda")

# merge the LoRA into the transformer (alpha = rank, so scale 1.0)
lora = load_file(hf_hub_download("Angais/MOSS-Image-PixelArt-16x16", "moss-image-pixelart-16x16.safetensors"))
with torch.no_grad():
    for key in [k for k in lora if k.endswith("lora_A.weight")]:
        layer = pipe.transformer.get_submodule(key[:-len(".lora_A.weight")])
        delta = lora[key.replace("lora_A", "lora_B")].float().cuda() @ lora[key].float().cuda()
        layer.weight.add_(delta.to(layer.weight.dtype))

prompt = "pixel art 16x16 game icon, rusty iron key with a red ribbon, dark outline, plain magenta background"
img = pipe(prompt, height=512, width=512, num_inference_steps=50, guidance_scale=3.0,
           generator=torch.Generator("cuda").manual_seed(0)).images[0]

rgb = np.asarray(img.convert("RGB"))[16::32, 16::32]                   # 16x16, one pixel per 32x32 block
alpha = np.where(np.abs(rgb.astype(int) - [255, 0, 255]).max(-1) <= 90, 0, 255).astype(np.uint8)
Image.fromarray(np.dstack([rgb, alpha]), "RGBA").save("icon.png")       # exact 16x16 sprite, transparent background

Upscale with nearest-neighbour (for example x8) to view it.

Results

Measured on 80 uncurated renders (64 varied prompts plus 16 test objects, 7 of them absent from the training set):

Raw 512x512 renders before the 16x16 conversion.

Training

• Data: 550 synthetic 16x16 icons (CC0), each upscaled x32 onto flat magenta, captioned with the prompt that made it.
• Setup: MOSS-Image-Base transformer in bf16, LoRA rank 32 / alpha 32 on the q, k, v, o and feed-forward linears of all 42 blocks (text encoder frozen), 512x512, batch 2, AdamW (lr 1e-4, 100 warm-up steps, linear decay), flow-matching loss, 5% caption dropout, 1,500 steps (about 15 minutes on one A100).

Limitations

• 16x16 icons only: one object, centred, no scenes or characters in poses.
• Slow compared to distilled models: 50 steps with guidance.
• Avoid asking for pure magenta objects: magenta is reserved for the background.
• Occasional odd shapes (a "black pirate hat" may grow straps underneath).
• Outputs are AI-generated. Where law or platform policy requires it, disclose that when you publish them.

License

Apache License 2.0, the same as MOSS-Image-Base (see LICENSE and NOTICE). Not affiliated with or endorsed by the OpenMOSS Team. A Krea 2 version of this LoRA, trained on the same data, is also available.