mingyi456/UnCanny-Photorealism-Chroma-DF11

Update: I have uploaded the latest v1.3 update to this model

For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11

Feel free to request for other models for compression as well, although models whose architecture I am unfamiliar with might be slightly tricky for me.

The original BF16 weights used for the compression were obtained from https://civitai.com/models/2086389/uncanny-photorealism-chroma

How to Use

diffusers

1. Install the DFloat11 pip package (installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed):

   pip install dfloat11[cuda12]
   # or if you have CUDA version 11:
   # pip install dfloat11[cuda11]
   

2. Download the uncannyPhotorealism_v12-DF11.safetensors file and place it in a local directory of your choice.

3. To use the DFloat11 model, run the following example code in Python:

   import torch
   from diffusers import ChromaPipeline, ChromaTransformer2DModel
   from dfloat11 import DFloat11Model
   from transformers.modeling_utils import no_init_weights
   pattern_dict = {
       r"distilled_guidance_layer\.layers\.\d+": (
           "linear_1",
           "linear_2"
       ),
       r"transformer_blocks\.\d+": (
           "attn.to_q",
           "attn.to_k",
           "attn.to_v",
           "attn.add_k_proj",
           "attn.add_v_proj",
           "attn.add_q_proj",
           "attn.to_out.0",
           "attn.to_add_out",
           "ff.net.0.proj",
           "ff.net.2",
           "ff_context.net.0.proj",
           "ff_context.net.2",
       ),
       r"single_transformer_blocks\.\d+": (
           "proj_mlp",
           "proj_out",
           "attn.to_q",
           "attn.to_k",
           "attn.to_v",
       ),
   }
   with no_init_weights():
     transformer = ChromaTransformer2DModel.from_config(
         ChromaTransformer2DModel.load_config(
             "lodestones/Chroma1-HD",
             subfolder="transformer"
         ),
         torch_dtype=torch.bfloat16
     ).to(torch.bfloat16)
   
   pipe = ChromaPipeline.from_pretrained(
       "lodestones/Chroma1-HD",
       transformer=transformer,
       torch_dtype=torch.bfloat16
   )
   DFloat11Model.from_single_file('uncannyPhotorealism_v12-DF11.safetensors', device='cpu', bfloat16_model=pipe.transformer, pattern_dict=pattern_dict) # Make sure to download the file first, and edit the filepath accordingly
   pipe.enable_model_cpu_offload()
   prompt = """professional cinematic film still. a beautiful punk woman in her early twenties. shaking her head and hair in the middle the crowd at a punk rock concert. her hair is all over the place with her movement. she is sweating. the woman has a narrow waist and small perky breasts. She has full lips, long curly blond hair, her hair is shaved on one side, freckles dust her nose.
   Wearing denim shorts and a crop top.
   A tattoo is visible on her chest above her top, it reads:
   "UnCanny"
   The photo is sharp and crisp, 8k - with a stylized cinematic atmosphere.
   """
   negative_prompt = "heavy bokeh, heavy film grain, jpeg artifacts, blurry, plasticy skin, undetailed skin, unsaturated tones, 3D image, CGI rendering, AI generated image, watermark, signature. undetailed bodies, undetailed faces, undetailed background. Anime, cartoon, manga, cel-shaded, stylized illustration, unrealistic proportions, painterly, sketch, illustration styles, CGI style, monochrome"
   image = pipe(
       prompt,
       guidance_scale=3.5,
       num_inference_steps=35,
       max_sequence_length=256,
       generator=torch.Generator("cpu").manual_seed(0)
   ).images[0]
   image.save("uncannyPhotorealism_v12-DF11.png")
   

ComfyUI

Refer to this model instead.