ControlNet v1.1 for Stable Diffusion 1.5 and 2.0

The models of Stable Diffusion 1.5 and Stable Diffusion 2.0 are compatible, which means that the model files of ControlNet v1.1 can also be used on Stable Diffusion 2.0.

This article is a compilation of different types of ControlNet models that support SD1.5 / 2.0, organized by ComfyUI-WIKI.

ControlNet v1.1

Large Size from lllyasviel

Provider: lllyasviel
GitHub Repository:https://github.com/lllyasviel/ControlNet-v1-1-nightly (opens in a new tab)
Hugging Face Repository:https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main (opens in a new tab) Compatible Stable Diffusion Versions: stable diffusion v1.5,2.0 Size: 1.45 GB

Medium - Safetensors/FP16 中等体积，来自 comfyanonymous 提供的版本

Provider：comfyanonymous Hugging Face Repository：https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main (opens in a new tab) Size: 723MB Note: Provided by the ComfyUI author's Hugging Face repository, it offers a better user experience in ComfyUI and is also compatible with other UIs.

Small - Control lora Safetensors/FP16 较小体积来自 comfyanonymous 提供的版本

Provider：comfyanonymous Hugging Face Repository：https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/tree/main (opens in a new tab) Size: 136MB Note: Provided by the ComfyUI author's Hugging Face repository, it offers a better user experience in ComfyUI and is also compatible with other UIs.

ControlNet for StableDiffusion 2.1

Provider: thibaud
Hugging Face Repository:https://huggingface.co/thibaud/controlnet-sd21 (opens in a new tab)

The following file formats are divided into .safetensors and .ckpt. Generally, if you only intend to use the corresponding model, downloading the .safetensors file will suffice.

How to Use ControlNet Model in ComfyUI

The usage of the ControlNet model is focused in the following article:

It will cover the following topics:

• How to install the ControlNet model in ComfyUI
• How to invoke the ControlNet model in ComfyUI
• ComfyUI ControlNet workflow and examples
• How to use multiple ControlNet models, etc.

Overview of ControlNet 1.1 Model

ControlNet 1.1 is an updated and optimized version based on ControlNet 1.0, with the same architecture. It includes all previous models and adds several new ones, bringing the total count to 14.

New Features and Improvements ControlNet 1.1 introduces several new features and improvements:

• New Soft Edges: A new soft edge processing method called "SoftEdge_safe" has been added to enhance the naturalness of image edges.
• Preprocessor Expansion: Multiple new preprocessors have been added, such as Canny, Depth, inpaint, etc., which can be used for various image processing tasks.
• Robustness and Quality Enhancement: The version mainly strengthens the model's robustness and the quality of the generated images.

Application Scenarios for ControlNet 1.1

ControlNet 1.1 can be applied to a variety of AI painting scenarios, including but not limited to:

Image Restoration: Utilize the inpaint preprocessor to fill and restore damaged or missing parts of an image. Edge Detection: Extract edge information from images using the Canny preprocessor for further application in other image processing tasks. Depth Map Generation: Generate depth information of images through the Depth preprocessor to create a more three-dimensional visual effect.