MOSS-Image colorize LoRA (v2)

Colorize v3 is in sorryhyun/MOSS-I2I-LORAs (colorize/), alongside the sanitize and decensor LoRAs and their ComfyUI workflows. This repo keeps v1 and v2.

Two of the twelve held-out pages, seed 1, from v2. Each panel is labelled: the B&W page (the input), the colour target, then v2 with the full colour caption, with a sparse caption, and with the page only.

• Left (touhou). Full caption (layout JSON, see Captions): {"summary":{"series":["touhou"],"caption":"purple eyes, purple hair, purple eyes, black hairband, blue shirt, green hair, yellow shirt, black hat"}}. Sparse: touhou, blue shirt, black hairband.
• Right (blue archive). Full caption: {"summary":{"series":["blue archive"],"caption":"black pantyhose, blue necktie, grey hair, purple eyes, white shirt, white skirt"}}. Sparse: blue necktie, white skirt, purple eyes.

A LoRA for MOSS-Image Base that colorizes a black-and-white manga page or illustration. The page goes in twice: through Qwen3.5's vision tower inside the text encoder, and as a clean VAE latent in the DiT sequence. The output keeps the page's line work, faces and lettering.

Status: research probe. It has been checked by eye on 12 held-out pages, one seed each. Base was not trained for image conditioning. This LoRA teaches it to read a reference.

What changed in v2

What this changes in use:

• Sparse captions work. Give the few colours you care about (blonde hair, blue bikini). Those are followed, and the rest is filled in plausibly.
• Page only. It now commits to colours. Early in training it fell back to grey and silver.
• Halftone. It turns dot screentone into flat colour.

v2 needs the updated ComfyUI-MOSS-i2i node, the version that reads ref_timestep from the LoRA metadata. An older node runs v2 with the ref at the step's timestep, which is off what it trained on. v1 still loads as before, because it has no ref_timestep in its metadata.

Captions

Besides the page alone, v2 was trained on two kinds of text. Use one of these:

• A few colour tags, plain: touhou, blue shirt, black hairband. The tag-dropped variants were plain tags. On a page with lettering, the variant also carried . OCR result as follows "…" [y0, x0, y1, x1], ….

• The full colour caption as layout JSON. This is how every full caption was trained:

  {"summary":{"series":["touhou"],"caption":"purple hair, blue shirt, green hair, yellow shirt"}}
  

  A page with lettering adds one item per line, boxes in the generated image's pixels with the origin at its centre (yxyx):

  {"summary":{...},"items":[{"label":"text","caption":"text","text":"カタ","bbox":[82,-44,114,-3]}],"bbox_coord_space":"abs_centralized","bbox_format":"yxyx"}
  

A long plain tag list (the first v2's full-caption format) was not a training format for this file and has not been tested with it. Put a long list in the layout JSON. The OCR text is optional: 41 % of training pages had no lettering.

Workflow

Install the custom node from here.

Drag and drop the following image to comfyui.

How it works

MOSS-Image has no image input. Its 4D RoPE (r, f, h, w) has an r axis that its training always set to 0. This LoRA was trained with:

• Text encoder input. The page through the vision tower, then the colour caption, in one user turn: <|im_start|>user\n<|vision_start|>[page]<|vision_end|>{caption}<|im_end|>\n<|im_start|>assistant\n<think>\n. At the target's pixel count, that is ~1000 image tokens at 1 MP. The vision tower alone brings back content but not line placement: one token per 32×32 px.
• VAE reference. The page at half size per side (1/4 area) goes into the DiT sequence as [image | ref | text].
  • The ref tokens get no noise. In v2 they also get their own block modulation from timestep 0, which is the modulation of a clean image.
  • On RoPE they sit at r = 1, with their (h, w) stretched onto the target grid, so each ref token lines up with the region it covers.
  • The ref goes into every CFG branch. The head and the loss read the image tokens only.

Files

ComfyUI

It needs two custom nodes:

• moss-image (OpenMOSS): the MODEL / CLIP / VAE loaders.
• ComfyUI-MOSS-i2i: three nodes for this LoRA.
  • MOSS i2i LoRA Loader. It reads ref_timestep from the metadata.
  • MOSS i2i Text Encode (page + text): the vision-tower encode above.
  • MOSS i2i Reference: the VAE ref, plus an empty latent at the target size.

Wiring (workflows/moss-i2i-colorize.json):

Load Image (page) ──┬─────────────────────────────┐
                    │                             │
MOSS-Image HF Loader ─ MODEL ─ i2i LoRA Loader ─ i2i Reference ─ ModelSamplingSD3(5) ─ SLG(single 22) ─┐
                     ─ CLIP ─┬ i2i Text Encode (page, colour tags) ── positive ─────────────────────────┤
                             └ CLIP Text Encode ("") ──────────────── negative ─────────────────────────┤ KSampler
                     ─ VAE ── i2i Reference ── LATENT ──────────────────────────────────────────────────┘

• Positive. The page plus text in one of the forms under Captions: a few plain colour tags with the copyright first (touhou, purple hair, blue shirt), or the full list as layout JSON. An empty text gives the page-only mode.
• Negative. A plain text encode of "".
• Resolution. Use the same value (1024) on Text Encode and Reference. The page is fit to ~1 MP at its native aspect, with sides multiples of 16.
• Sampler. CFG 3, with block 22 skipped on the uncond branch, and shift 5. The workflow uses ER-SDE at 30 steps, the setting the eval below used. The MOSS Spectrum KSampler (ComfyUI-Spectrum-KSampler) works too.

Training (v2)