Controlnet V1 1 Sd15 Sd2

ControlNet v1.1 for Stable Diffusion 1.5 and 2.0

The models of Stable Diffusion 1.5 and Stable Diffusion 2.0 are compatible, which means that the model files of ControlNet v1.1 can also be used on Stable Diffusion 2.0.

This article is a compilation of different types of ControlNet models that support SD1.5 / 2.0, organized by ComfyUI-WIKI.

ControlNet v1.1

Large Size from lllyasviel

Provider: lllyasviel
GitHub Repository:https://github.com/lllyasviel/ControlNet-v1-1-nightly (opens in a new tab)
Hugging Face Repository:https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main (opens in a new tab) Compatible Stable Diffusion Versions: stable diffusion v1.5,2.0 Size: 1.45 GB

File NameSizeUpdate TImeNoteDownload Links
control_v11e_sd15_ip2p.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11e_sd15_shuffle.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11f1e_sd15_tile.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11f1p_sd15_depth.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_canny.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_inpaint.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_lineart.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_mlsd.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_normalbae.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_openpose.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_scribble.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_seg.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15_softedge.pth1.45 GBMay 2023Download Link (opens in a new tab)
control_v11p_sd15s2_lineart_anime.pth1.45 GBMay 2023Download Link (opens in a new tab)

Medium - Safetensors/FP16 中等体积,来自 comfyanonymous 提供的版本

Provider:comfyanonymous Hugging Face Repositoryhttps://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main (opens in a new tab) Size: 723MB Note: Provided by the ComfyUI author's Hugging Face repository, it offers a better user experience in ComfyUI and is also compatible with other UIs.

File NameSizeUpdate TimeDownload Links
control_v11e_sd15_ip2p_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11e_sd15_shuffle_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11f1e_sd15_tile_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11f1p_sd15_depth_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_canny_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_inpaint_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_lineart_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_mlsd_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_normalbae_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_openpose_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_scribble_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_seg_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15_softedge_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11p_sd15s2_lineart_anime_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)
control_v11u_sd15_tile_fp16.safetensors723 MB2023年4月Download Link (opens in a new tab)

Small - Control lora Safetensors/FP16 较小体积来自 comfyanonymous 提供的版本

Provider:comfyanonymous Hugging Face Repositoryhttps://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main (opens in a new tab) Size: 136MB Note: Provided by the ComfyUI author's Hugging Face repository, it offers a better user experience in ComfyUI and is also compatible with other UIs.

File NameSizeUpdate TimeDownload Links
control_lora_rank128_v11e_sd15_ip2p_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11e_sd15_shuffle_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11f1e_sd15_tile_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11f1p_sd15_depth_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_canny_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_inpaint_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_lineart_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_mlsd_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_normalbae_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_openpose_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_scribble_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_seg_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15_softedge_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)
control_lora_rank128_v11p_sd15s2_lineart_anime_fp16.safetensors136 MB2023年4月Download Link (opens in a new tab)

ControlNet for StableDiffusion 2.1

Provider: thibaud
Hugging Face Repository:https://huggingface.co/thibaud/controlnet-sd21 (opens in a new tab)

The following file formats are divided into .safetensors and .ckpt. Generally, if you only intend to use the corresponding model, downloading the .safetensors file will suffice.

File NameSizeUpdate TimeNoteDownload Link
control_v11p_sd21_ade20k.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_canny.ckpt9.59 GBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_canny.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_color.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_depth.ckpt9.59 GBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_depth.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_hed.ckpt9.59 GBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_hed.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_lineart.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_normalbae.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_openpose.ckpt9.59 GBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_openpose.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_openposev2.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_scribble.ckpt9.59 GBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_scribble.safetensors728 MBApril 2023Download Link (opens in a new tab)
control_v11p_sd21_zoedepth.safetensors728 MBApril 2023Download Link (opens in a new tab)

How to Use ControlNet Model in ComfyUI

The usage of the ControlNet model is focused in the following article:

How to use ControlNet in ComfyUI

It will cover the following topics:

  • How to install the ControlNet model in ComfyUI
  • How to invoke the ControlNet model in ComfyUI
  • ComfyUI ControlNet workflow and examples
  • How to use multiple ControlNet models, etc.

Overview of ControlNet 1.1 Model

ControlNet 1.1 is an updated and optimized version based on ControlNet 1.0, with the same architecture. It includes all previous models and adds several new ones, bringing the total count to 14.

New Features and Improvements ControlNet 1.1 introduces several new features and improvements:

  • New Soft Edges: A new soft edge processing method called "SoftEdge_safe" has been added to enhance the naturalness of image edges.
  • Preprocessor Expansion: Multiple new preprocessors have been added, such as Canny, Depth, inpaint, etc., which can be used for various image processing tasks.
  • Robustness and Quality Enhancement: The version mainly strengthens the model's robustness and the quality of the generated images.

Application Scenarios for ControlNet 1.1

ControlNet 1.1 can be applied to a variety of AI painting scenarios, including but not limited to:

Image Restoration: Utilize the inpaint preprocessor to fill and restore damaged or missing parts of an image. Edge Detection: Extract edge information from images using the Canny preprocessor for further application in other image processing tasks. Depth Map Generation: Generate depth information of images through the Depth preprocessor to create a more three-dimensional visual effect.