Rebalance Model Overview

Model Overview

Rebalance is a high-fidelity image generation model trained on a curated dataset comprising thousands of cosplay photographs and handpicked, high-quality real-world images. All training data was sourced exclusively from publicly accessible internet content, and the dataset explicitly excludes any NSFW material. The primary goal of Rebalance is to produce photorealistic outputs that overcome common AI artifacts—such as an oily, plastic, or overly flat appearance—delivering images with natural texture, depth, and visual authenticity.

Training Strategy

Training was conducted in multiple stages, broadly divided into two phases:

• Cosplay Photo Training
  Focused on refining facial expressions, pose dynamics, and overall human figure realism—particularly for female subjects.
• High-Quality Photograph Enhancement
  Aimed at elevating atmospheric depth, compositional balance, and aesthetic sophistication by leveraging professionally curated photographic references.

Captioning & Metadata

The model was trained using two complementary caption formats: plain text and structured JSON. Each data subset employed a tailored JSON schema to guide fine-grained control during generation.

For cosplay images, the JSON includes:

{
  "caption": "...",
  "image_type": "...",
  "image_style": "...",
  "lighting_environment": "...",
  "tags_list": [...],
  "brightness": number,
  "brightness_name": "...",
  "hpsv3_score": score,
  "aesthetics": "...",
  "cosplayer": "anonymous_id"
}

Note: Cosplayer names are anonymized (using placeholder IDs) solely to help the model associate multiple images of the same subject during training—no real identities are preserved.

For high-quality photographs, the JSON structure emphasizes scene composition:

{
  "subject": "...",
  "foreground": "...",
  "midground": "...",
  "background": "...",
  "composition": "...",
  "visual_guidance": "...",
  "color_tone": "...",
  "lighting_mood": "...",
  "caption": "..."
}

In addition to structured JSON, all images were also trained with plain-text captions and with randomized caption dropout (i.e., some training steps used no caption or partial metadata). This dual approach enhances both controllability and generalization.

Inference Guidance

• For maximum aesthetic precision and stylistic control, use the full JSON format during inference.
• For broader generalization or simpler prompting, plain-text captions are recommended.

Technical Details

All training was performed using lrzjason/T2ITrainer, a customized extension of the Hugging Face Diffusers DreamBooth training script. The framework supports advanced text-to-image architectures, including Qwen and Qwen-Edit (2509).

Previous Work

This project builds upon several prior tools developed to enhance controllability and efficiency in diffusion-based image generation and editing:

• ComfyUI-QwenEditUtils
  A collection of utility nodes for Qwen-based image editing in ComfyUI, enabling multi-reference image conditioning, flexible resizing, and precise prompt encoding for advanced editing workflows.
• ComfyUI-LoraUtils
  A suite of nodes for advanced LoRA manipulation in ComfyUI, supporting fine-grained control over LoRA loading, layer-wise modification (via regex and index ranges), and selective application to diffusion or CLIP models.
• T2ITrainer
  A lightweight, Diffusers-based training framework designed for efficient LoRA (and LoKr) training across multiple architectures—including Qwen Image, Qwen Edit, Flux, SD3.5, and Kolors—with support for single-image, paired, and multi-reference training paradigms.

These tools collectively establish a robust ecosystem for training, editing, and deploying personalized diffusion models with high precision and flexibility.

Contact

Feel free to reach out via any of the following channels:

• Twitter: @Lrzjason
• Email: lrzjason@gmail.com
• QQ Group: 866612947
• WeChat ID: fkdeai
• CivitAI: xiaozhijason

Sponsors me for more open source projects: