Skip to content
ComfyUI tutorial ComfyUI Advanced Tutorial1.2 SD1.5 Depth ControlNet

Complete Guide - How to Use Depth ControlNet with SD1.5 in ComfyUI

Introduction to SD1.5 Depth ControlNet

SD1.5 Depth ControlNet Comparison

Depth ControlNet is a specialized ControlNet model designed to control image depth and spatial structure. By analyzing the depth information of input images, it helps AI maintain correct spatial relationships and perspective effects when generating new images. This model is particularly useful in interior design, architectural design, and scene reconstruction as it can accurately understand and preserve spatial depth information.

This tutorial focuses on the usage and techniques of the Depth ControlNet model for SD1.5. We will cover other versions and types of ControlNet models in future tutorials.

Key Features of Depth ControlNet

  • Spatial Control: Precise control over spatial depth and perspective relationships
  • Scene Reconstruction: Maintains original spatial layout while changing style and content
  • Interior Design: Particularly suitable for interior scene redesign and style transformation
  • Architectural Visualization: Highly effective for 3D visualization of architectural and interior designs
  • Product Showcase: Ideal for creating product displays with depth perception
  • Scene Planning: Assists in landscape design and urban planning visualization

SD1.5 Depth ControlNet Workflow Preparation

1. Installing Required Plugins

Since ComfyUI Core doesn’t come with the corresponding Depth image preprocessor, you’ll need to download the preprocessor plugin first. This tutorial requires the ComfyUI ControlNet Auxiliary Preprocessors plugin to generate depth maps.

We recommend using ComfyUI Manager for installation. For detailed plugin installation instructions, refer to ComfyUI Plugin Installation Guide

The latest version of ComfyUI Desktop comes with ComfyUI Manager pre-installed

Method 1: Using ComfyUI Manager (Recommended)

  1. First install ComfyUI Manager
  2. Search for and install “ComfyUI ControlNet Auxiliary Preprocessors” in the Manager

Method 2: Installation via Git

  1. Open command line and cd into ComfyUI’s custom_nodes directory
  2. Execute the following commands:
git clone https://github.com/Fannovel16/comfyui_controlnet_aux
cd comfyui_controlnet_aux
pip install -r requirements.txt

Note: Restart ComfyUI after installing plugins

Method 3: Manual Installation (Not Recommended)

  1. Visit https://github.com/Fannovel16/comfyui_controlnet_aux
  2. Download the ZIP file of the repository code
  3. Extract and copy the files to the ComfyUI/custom_nodes/ folder

2.1 Download Required Models

You’ll need to install the following models:

Model TypeModel FileDownload Link
SD1.5 Base Modeldreamshaper_8.safetensorsCivitai
Depth ControlNet Modelcontrol_v11f1p_sd15_depth.pthHugging Face
VAE Model (Optional)vae-ft-mse-840000-ema-pruned.safetensorsHugging Face

While you can use any SD1.5 model you have, this tutorial uses dreamshaper_8 as an example. For interior design and similar applications, you should choose models specifically optimized for interior or architectural design.

2.2 Model File Structure

Place the model files according to this structure:

📁ComfyUI
├── 📁models
│   ├── 📁checkpoints
│   │   └── 📁SD1.5
│   │       └── dreamshaper_8.safetensors
│   ├── 📁controlnet
│   │   └── 📁SD1.5
│   │       └── control_v11f1p_sd15_depth.pth
│   └── 📁vae
│       └── vae-ft-mse-840000-ema-pruned.safetensors

3. Workflow Files

Interior design is one of the most common applications for Depth ControlNet. Through depth information control, you can maintain the original spatial layout while completely transforming the interior style and atmosphere.

Here’s an example of converting a traditional living room into a cyberpunk style:

Input Image Output Result

SD1.5 Depth ControlNet Workflow Guide

Main Components

This workflow uses the following key nodes:

  1. LoadImage: Loads the input image
  2. Zoe-DepthMapPreprocessor: Generates depth maps, provided by the ComfyUI ControlNet Auxiliary Preprocessors plugin
    • resolution: Controls the depth map resolution, affecting its detail level:
      • Higher resolution (e.g., 768, 1024):
        • Pros: Captures more details, better for complex interiors and architecture
        • Cons: Slower processing, higher VRAM usage
        • Use cases: Detailed interior design, architectural detail reproduction
      • Lower resolution (e.g., 384, 512):
        • Pros: Faster processing, lower VRAM usage
        • Cons: May lose some detail information
        • Use cases: Quick previews, simple scene reconstruction
      • Recommended settings:
        • General scenes: 512 is a good balance
        • High detail requirements: 768 or higher
        • Quick testing: 384
    • Uses Zoe depth estimation algorithm for high-quality depth maps
    • Particularly suitable for interior and architectural scenes
    • Can preview generated depth maps through PreviewImage node

Tip: Start with lower resolution for testing and adjustment, then increase resolution for final generation

Tip: Zoe-DepthMapPreprocessor is currently one of the best depth map generators for architectural and interior scenes, handling complex spatial structures and details well

Depth ControlNet Parameter Guide and Recommendations

KSampler Parameters

  1. steps

    • Purpose: Controls generation detail and quality
    • Recommended range: 25-30 steps
    • Details:
      • Higher steps (30+): More detailed results but slower
      • Lower steps (20-): Faster but may lose details
      • Recommend minimum 25 steps for accurate spatial structure

  2. cfg (Classifier Free Guidance)

    • Purpose: Controls adherence to prompts
    • Recommended range: 6-8
    • Details:
      • Higher values (8+): Stricter prompt adherence but less creative
      • Lower values (5-): More creative but may deviate from intent
      • Recommend 6-7 for interior design to balance accuracy and creativity

  3. sampler_name

    • Recommendations:
      • euler: Fast, good for previews
      • dpmpp_2m: High quality, ideal for final generation
    • Scene-specific choices:
      • Interior design: Prefer dpmpp_2m
      • Quick tests: Use euler

  4. scheduler

    • Recommended: karras
    • Reason: Best for maintaining spatial structure

ControlNet Control Parameters

  1. strength

    • Purpose: Determines depth information influence
    • Recommended range: 0.8-1.0
    • Scene recommendations:
      • Interior design: 0.9-1.0
      • Building exteriors: 0.8-0.9
      • Simple scenes: 0.7-0.8

  2. start_percent and end_percent

    • Purpose: Controls depth guidance timing
    • Default settings:
      • start_percent: 0 (apply from start)
      • end_percent: 1 (maintain throughout)
    • Special cases:
      • For more creative freedom: set end_percent to 0.8-0.9
      • For strict spatial structure: keep defaults

Tip: Combine parameters effectively - for interior design, use high strength (0.9+) with high steps (30+) for best results

Workflow Node Description

Main node connections:

  1. Input Section:

    • LoadImage → Zoe-DepthMapPreprocessor → PreviewImage (for depth map preview)
    • LoadImage → Zoe-DepthMapPreprocessor → ControlNetApplyAdvanced

  2. Model Loading Section:

    • CheckpointLoaderSimple (loads base model)
    • ControlNetLoader (loads Depth ControlNet)

  3. Prompt Processing Section:

    • CLIPTextEncode (positive prompts)
    • CLIPTextEncode (negative prompts)

  4. Generation Control Section:

    • KSampler (controls generation process)
    • VAEDecode (converts latent to final image)

Depth ControlNet Tips and Best Practices

  1. Depth Map Quality Control

    • Use high-quality input images
    • Ensure clear spatial hierarchy
    • Avoid overly complex scenes
    • Consider lighting’s effect on depth maps

  2. Prompt Writing

    • Detail spatial relationships
    • Include material and lighting information
    • Specify important depth elements
    • Use professional terminology
    • Recommended keywords:
      • Spatial: depth, perspective, spatial layout, composition
      • Quality: professional, high quality, detailed, realistic
      • Style: modern, minimalist, futuristic (as needed)

  3. Troubleshooting

    • Weak spatial feel: Increase strength
    • Lost details: Lower cfg
    • Structure distortion: Increase steps
    • Inaccurate depth: Adjust resolution
    • Wrong style: Optimize prompts

FAQ

  1. Why is spatial sense weak in generated images?

    • Check depth map clarity
    • Verify strength value
    • Consider increasing steps

  2. How to improve image quality?

    • Use higher resolution
    • Select appropriate sampler
    • Optimize prompts

  3. How to speed up generation?

    • Lower resolution
    • Use faster sampler
    • Reduce steps

  4. How to maintain original layout?

    • Increase strength
    • Keep end_percent at 1
    • Use detailed spatial descriptions

Related Resources