Detailed Guide to Flux ControlNet Workflow

This tutorial will guide you on how to use Flux’s official ControlNet models in ComfyUI. We will cover the usage of two official control models: FLUX.1 Depth and FLUX.1 Canny.

This tutorial is based on and updated from the ComfyUI Flux examples

Model Introduction

FLUX.1 Depth [dev]

• 12 billion parameter rectified flow transformer model
• Structure guidance based on depth maps
• Uses guided distillation training for improved efficiency
• Supports personal, research, and commercial use

FLUX.1 Canny [dev]

• 12 billion parameter rectified flow transformer model
• Structure guidance based on Canny edge detection
• Also uses guided distillation training method
• Follows FLUX.1 [dev] non-commercial license

Model Version Overview

Flux ControlNet models are available in two versions: Full Model and LoRA Model.

Full Model Version

• Complete model file containing all weights
• Requires larger VRAM
• Best generation quality

LoRA Version

• Lightweight model containing only difference weights
• Requires base Flux model
• Lower VRAM usage

Preparation

1. Update ComfyUI

First, ensure your ComfyUI is updated to the latest version. If you don’t know how to update and upgrade ComfyUI, please refer to How to Update and Upgrade ComfyUI.

Note: Flux ControlNet functionality requires the latest version of ComfyUI, so please complete the update first.

2. Full Version Model Downloads

3. LoRA Version Model Downloads

4. Workflow File Downloads

5. System Requirements

• VRAM Requirements: Recommended minimum 16GB VRAM
• If VRAM is limited, you can use fp8 version models to reduce VRAM usage

Flux ControlNet Workflow Usage Guide

Full Version Usage Tips

1. ControlNetLoader

   • Depth Model: For maintaining reference image 3D structure
   • Canny Model: For maintaining reference image edge lines

2. Preprocessing Nodes

   • DepthPreprocessor: Generates depth maps, no additional adjustment needed
   • CannyEdgePreprocessor: Threshold adjustments control edge detection detail
     • low threshold: Edge detection sensitivity
     • high threshold: Edge line completeness

3. FluxGuidance

   • Recommended value: 3.5-4.0
   • Higher values make generation results closer to prompt description

4. ModelSamplingFlux

   • shift_factor: 1.15 (default value, usually no adjustment needed)
   • multiplier: 0.5 (can be fine-tuned as needed)

LoRA Version Usage Tips

1. LoraLoaderModelOnly Node

   • Load corresponding LoRA model file
   • Recommended strength parameter: 1.0
   • Requires base Flux model to be loaded first

2. InstructPixToPixConditioning Node

   • Used for image-to-image condition control
   • Requires VAE and original image input connections

3. KSampler Settings

   • Recommended steps: 20
   • Sampler: euler
   • Scheduler: normal
   • Denoise: 1.0

Version Selection Guidelines

1. When to Choose Full Version

   • Have sufficient VRAM (16GB+)
   • Seeking best generation quality
   • Need more precise control

2. When to Choose LoRA Version

   • Limited VRAM (8-16GB)
   • Need faster generation speed
   • Quality requirements are not extremely strict

Parameter Tuning Tips

Depth Control

• Suitable for scenes requiring spatial structure maintenance
• Recommended control strength: 0.6-0.9

Canny Control

• Suitable for scenes requiring line and contour maintenance
• Recommended control strength: 0.5-0.8

General Tips

• Test with default parameters first
• Gradually adjust control strength based on generation results
• Further optimize results by adjusting prompts

Advanced Applications

1. Mixed Control

   • Can use Depth and Canny control simultaneously
   • Adjust weights of different control networks

2. Style Transfer

   • Combine with LoRA models
   • Use style prompts

3. Batch Processing

   • Use batch processing nodes
   • Optimize workflow

Reference Resources

• FLUX.1 Depth Dev Official Page
• FLUX.1 Canny Dev Official Page
• ComfyUI Flux Examples
• blackforestlabs Blog