Shakker Labs Releases FLUX.1-dev-ControlNet-Union-Pro-2.0
Shakker Labs has recently released a new version of the ControlNet network for the FLUX.1-dev model - FLUX.1-dev-ControlNet-Union-Pro-2.0. This new model has been optimized in multiple aspects, especially in enhancing control effects and reducing model size.
Key Updates in the New Version
Compared to the previous FLUX.1-dev-ControlNet-Union-Pro, the new version has the following significant improvements:
- Smaller Model Size: Removed the mode embedding feature, reducing the model size from 6.15GB to 3.98GB
- Improved Control Effects: Optimized canny edge detection and pose control, providing better control precision and aesthetic effects
- Control Mode Adjustments: Added support for soft edge detection but removed support for tile mode
Online Experience
Supported Control Modes
This ControlNet model supports multiple control modes, including:
- Canny edge detection
- Soft Edge
- Depth
- Pose
- Gray
Users can use this model like a regular ControlNet, and it can be combined with other ControlNet models to achieve multiple control effects.
Model Showcase
Here are demonstrations of the model in various control modes:
Recommended Parameter Settings
The official recommendation for different control types includes the following parameter settings. You can adjust the controlnet_conditioning_scale and control_guidance_end parameters to achieve better control effects and detail preservation:
- Canny Edge: Using cv2.Canny algorithm, controlnet_conditioning_scale=0.7, control_guidance_end=0.8
- Soft Edge: Using AnylineDetector, controlnet_conditioning_scale=0.7, control_guidance_end=0.8
- Depth: Using depth-anything, controlnet_conditioning_scale=0.8, control_guidance_end=0.8
- Pose: Using DWPose, controlnet_conditioning_scale=0.9, control_guidance_end=0.65
- Gray: Using cv2.cvtColor, controlnet_conditioning_scale=0.9, control_guidance_end=0.8
For better generation stability, it is strongly recommended to use detailed prompts. In some cases, using multiple condition controls will yield better results.
Technical Details
This ControlNet model consists of 6 double blocks and 0 single blocks, with mode embedding removed. The model was trained from scratch for 300,000 steps at 512x512 resolution using a dataset of 20 million high-quality general and portrait images, with BFloat16 precision, batch size of 128, learning rate of 2e-5, guidance sampling range from 1 to 7, and text dropout rate of 0.20.
Related Resources
- InstantX/FLUX.1-dev-IP-Adapter
- InstantX/FLUX.1-dev-Controlnet-Canny
- Shakker-Labs/FLUX.1-dev-ControlNet-Depth
- Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro