ComfyUI tutorial
ComfyUI Basic Tutorial
7.Inpaint

How to inpainting Image in ComfyUI?

Image partial redrawing refers to the process of regenerating or redrawing the parts of an image that you need to modify.

In the ComfyUI Github repository partial redrawing workflow example, you can find examples of partial redrawing. However, in my use, the effect of using the VAE Internal Patch Encoder is not very good. This article compares the VAE Internal Patch Encoder with the Laton Noise Mask and demonstrates the corresponding example effects.

At the moment, I haven't produced an English version of the tutorial video, so please directly download the workflow file for viewing and learning.

Original tutorial address | Chinese (opens in a new tab)

Preparation Phase: Prerequisites for Partial Redrawing

Firstly, we need to mark the area on the image that needs to be modified, which is achieved by drawing a mask. The role of the mask is to inform the model of the specific location we wish to modify.

Drawing the Mask

At the beginning of image editing, it is necessary to specify the area that needs to be modified. By drawing a mask, the model is pointed out the specific scope for modification.

Updating Workflow Nodes

Since the original VAE encoder does not support masks, it is necessary to choose a node that can integrate masks. This article recommends using the VAE Internal Patch Encoder and Laton Noise Mask, which have their own characteristics in implementation.

Implementation Process: Detailed Steps for Partial Redrawing

Step One: Image Loading and Mask Drawing

  • Import the image at the Load Image node.
  • Use the mask tool to draw on specific areas, then use it for input to subsequent nodes for redrawing.

Step Two: Building the ComfyUI Partial Redrawing Workflow

For the specific workflow, please download the workflow file attached to this article and run it.

  • Select the image and mask in the latent node and connect it to the K Sampler.
  • Adjust the prompt words, such as changing "cat" to "dog", and set a new random seed.
  • Perform image generation and observe the changes in the results.

Step Three: Comparing the Effects of Two ComfyUI Nodes for Partial Redrawing

  • Apply the VAE Encode For Inpaint and Set Latent Noise Mask for partial redrawing.
  • Compare the performance of the two techniques at different denoising values.
    • The VAE Encode For Inpaint may cause the content in the masked area to be distorted at a low denoising value.
    • The Set Latent Noise Mask can maintain a high degree of similarity with the original image at a low denoising value.
  • Discuss the application scenarios and advantages of the two techniques.
    • The VAE Encode For Inpaint is suitable for creating new content that is unrelated to the original image.
    • The Set Latent Noise Mask is suitable for making local adjustments while retaining the characteristics of the original image, such as replacing the type of animal.