PixText

English | 简体中文

PixText refines local text in pixel space. Provide an image, the regions to edit, and the target text. The model generates those regions using the image as context and preserves pixels outside the actual edited footprints. Multiple targets are generated jointly.

MeanFlow is the default quick-start choice. Each model package contains its weights, configuration, fonts and SHA-256 checksums. MeanFlow needs no additional text encoder, OCR model, VAE, or teacher.

Example

Two input images and their targets are bundled in examples/. Cyan boxes mark the edited regions. Click an image to enlarge it.

Each row pairs two targets; every target shows input, MeanFlow output and Teacher output.

English tagline repair. examples/english_case repairs three damaged spots in a printed tagline: a broken letter plus two words, leaving the surrounding logo untouched.

Each row shows one repaired region: input, MeanFlow output and Teacher output.

Both cases use seed 42, and the adjacent JSON files record the reproduction metadata. To reproduce them, run the code from ATH-MaaS/PixText with --model pointing at the packages in models/.

Full-resolution images: pipeline input, MeanFlow, Teacher; tagline input, MeanFlow, Teacher.

Method

Installation

The inference code is at github.com/ATH-MaaS/PixText. Use Python 3.11 or newer and a CUDA GPU. Install a CUDA build of PyTorch and torchvision appropriate for your machine, then install the package:

pip install "git+https://github.com/ATH-MaaS/PixText.git"

Pillow must support RAQM. The system must provide HarfBuzz (libharfbuzz0 on Debian/Ubuntu). Font files are loaded from the model packages in this directory; no system font installation is required.

What is in this release

This directory contains both model packages, their fonts, example images and checksums. The inference code lives at github.com/ATH-MaaS/PixText.

Verify the release from its root:

sha256sum -c manifest.sha256

The benchmark for this model, TextRefinementBench, ships as a separate dataset package with its own card.

Model packages

Both packages are included under models/, laid out as follows:

models/
  PixText-Teacher/
  PixText-MeanFlow/
    model.safetensors
    config.json
    inference_config.json
    manifest.json
    README.md
    fonts/
      AlibabaPuHuiTi-3-55-Regular.ttf
      GoNotoCJKCore-v7.0.ttf
      OFL-GoNoto.txt
      README.md

Teacher uses the same package layout. model_index.json lists the model identifiers and weight checksums. from_pretrained loads a local directory and checks the files listed in its manifest, including the weights, configuration and fonts. A missing or modified bundled font is reported before weights are loaded. Complete packages can be moved without changing their configuration.

Python

from pixtext import PixTextPipeline, TextEdit

pipeline = PixTextPipeline.from_pretrained("models/PixText-MeanFlow")
result = pipeline.edit(
    "input.png",
    edits=[TextEdit(bbox=(120, 80, 160, 125), text="鹿")],
    seed=42,
)
result.image.save("output.png")

Use models/PixText-Teacher to load Teacher. Both variants can coexist in one process. Results contain image and metadata, including model identity, seed, sample identity, sampling settings, and crop geometry.

Command line

Create edits.json:

[{"bbox": [120, 80, 160, 125], "text": "鹿"}]

pixtext --model models/PixText-MeanFlow \
  --image input.png --edits edits.json --output output.png --seed 42

The CLI saves a PNG and a matching JSON metadata file. python -m pixtext provides the same command. --steps and --guidance-scale override the package defaults.

Batch editing

results = pipeline.edit_batch(
    ["first.png", "second.png"],
    [
        [TextEdit((120, 80, 160, 125), "鹿")],
        [TextEdit((20, 30, 100, 70), "PixText")],
    ],
    seeds=[42, 43],
    sample_ids=["first", "second"],
)

Inputs and pixel preservation

• Inputs are Pillow images or local image paths, converted to upright RGB using EXIF orientation. Coordinates refer to that upright image.
• Boxes use original-image pixel coordinates (x0, y0, x1, y1) and half-open intervals. Invalid or out-of-image boxes are rejected.
• Supply one character or a text line per target.
• Crops align to 16-pixel patches. Pixels outside those footprints remain identical to the original RGB image.
• Arbitrary image dimensions are supported.
• Targets are processed in input order, up to 12 per pass. For overlapping aligned crops, the last target written wins. An empty edit list returns the original RGB image.
• The renderer selects the first bundled font covering the entire target string, using PuHuiTi then Go Noto. Unsupported text produces a coverage error instead of missing-glyph squares.
• Editing requires CUDA FlexAttention.

Glyph-present and glyph-absent predictions provide guidance on the open interval (0.1, 1.0). At the default guidance setting, four MeanFlow updates use seven model forwards. Initial execution also includes compilation overhead.

License

Model weights and documentation are released under the Apache License 2.0; see LICENSE. The inference code is at ATH-MaaS/PixText. Bundled fonts keep their own terms, described in each model package's fonts/ directory.