How to Use Canny ControlNet SD1.5 Model in ComfyUI - Complete Guide

Introduction to SD1.5 Canny ControlNet

Canny ControlNet is one of the most commonly used ControlNet models. It uses the Canny edge detection algorithm to extract edge information from images, then uses this edge information to guide AI image generation.

This tutorial focuses on using the Canny ControlNet model with SD1.5

Related content: ControlNet Models Installation and Usage Guide ControlNet Models Download Links

Key Features of Canny ControlNet

• Structure Preservation: Effectively maintains the basic structure and outlines of the original image
• High Flexibility: Control guidance strength through edge detection parameter adjustments
• Wide Application: Suitable for sketches, line art, architectural designs, and various other scenarios
• Stable Results: Provides more stable and predictable guidance compared to other ControlNet models

Preparation for This Tutorial

1. Update ComfyUI and Install Required Models

Since some nodes use new ComfyUI nodes, you need to update ComfyUI to the latest version first

• For updating ComfyUI, please refer to ComfyUI Update Tutorial

First, you need to install the following models:

2. Model File Placement

Please place the model files according to the following structure:

📁ComfyUI
├── 📁models
│   ├── 📁checkpoints
│   │   └── 📁SD1.5
│   │       └── dreamshaper_8.safetensors
│   ├── 📁controlnet
│   │   └── 📁SD1.5
│   │       └── control_v11p_sd15_canny.pth
│   └── 📁vae
│       └── vae-ft-mse-840000-ema-pruned.safetensors

3. Download SD1.5 Canny ControlNet Workflow File

Save the image below locally, then load it into the LoadImage node after importing the workflow

Workflow Overview

This workflow consists of the following main parts:

1. Model Loading: Loading SD model, VAE model and ControlNet model
2. Prompt Encoding: Processing positive and negative prompts
3. Image Processing: Including image loading and Canny edge detection
4. ControlNet Control: Applying edge information to the generation process
5. Sampling and Saving: Generating and saving the final image

Key Nodes Explanation

1. LoadImage: Used to load input images

2. Canny: Performs edge detection with two important parameters:

   • low_threshold: Lower threshold, controls edge detection sensitivity
   • high_threshold: Upper threshold, controls edge continuity

3. ControlNetLoader: Loads the ControlNet model

4. ControlNetApplyAdvanced: Controls how ControlNet is applied, with parameters including:

   • strength: Control intensity
   • start_percent: When the influence begins
   • end_percent: When the influence ends

Usage Steps

1. Import Workflow

   • Download the workflow file from this tutorial
   • Click “Load” in ComfyUI, or drag and drop the downloaded JSON file into ComfyUI

2. Prepare Input Image

   • Prepare an image you want to process
   • Load the image using the LoadImage node

3. Adjust Canny Parameters

   • Recommended low_threshold range: 0.2-0.5
   • Recommended high_threshold range: 0.5-0.8
   • Preview edge detection results using the PreviewImage node

4. Set Generation Parameters

   • In the KSampler node:
     • steps: Recommended 20-30
     • cfg: Recommended 7-8
     • sampler_name: Recommended “dpmpp_2m”
     • scheduler: Recommended “karras”

5. Adjust ControlNet Strength

   • strength: 1.0 means fully following edge information
   • Reduce strength value as needed to weaken control

Tips and Recommendations

1. Edge Detection Parameter Adjustment

   • If too many edges: Increase threshold values
   • If too few edges: Decrease threshold values
   • Preview effects through PreviewImage node first

2. Prompt Writing

   • Positive prompts should detail desired style and details
   • Negative prompts should include elements to avoid
   • Prompts should relate to original image content

3. Common Issues Solutions

   • If generated image is too blurry: Increase cfg value
   • If edge following is insufficient: Increase strength value
   • If lacking details: Increase steps value

Practical Examples

Here are some common use cases and their parameter settings:

1. Line Art Coloring

   • low_threshold: 0.2
   • high_threshold: 0.5
   • strength: 1.0
   • steps: 25

2. Structure Redrawing

   • low_threshold: 0.4
   • high_threshold: 0.7
   • strength: 0.8
   • steps: 30

Related Resources

ControlNet Models Download More ControlNet Tutorials