InstantX Releases FLUX.1-dev IP-Adapter Model

InstantX team officially open-sourced the FLUX.1-dev IP-Adapter model on November 22, 2024. This is an IP-Adapter implementation based on FLUX.1-dev, bringing powerful image reference capabilities to the FLUX model.

FLUX.1-dev IP-Adapter Key Features

• Uses google/siglip-so400m-patch14-384 as image encoder
• Implements simple dual-layer linear MLPProjModel for projection
• Image token count set to 128
• Trained on 10 million open-source datasets with batch size 128 for 80,000 steps

Technical Implementation

The model adds new layers in 38 single-layer blocks and 19 dual-layer blocks. It utilizes google/siglip-so400m-patch14-384 for image encoding, chosen primarily for its outstanding performance.

Usage Methods

Currently supports two main usage methods:

1. Direct calling through Python code
2. Using the ComfyUI-IPAdapter-Flux plugin in ComfyUI

ComfyUI Workflow

InstantX provides example workflow files for immediate use:

• Example Workflow File

Online Experience

Users can experience FLUX.1-dev-IP-Adapter through the following platforms:

• Shakker AI Platform
• Shakker Generator
• Online ComfyUI

Open Source Resources

Related resources are now open-sourced:

• Model Download: FLUX.1-dev-IP-Adapter
• ComfyUI Plugin: ComfyUI-IPAdapter-Flux

Usage Limitations

The model currently has some limitations:

• Not suitable for fine-grained style transfer or character consistency
• Trade-offs between content leakage and style transfer
• Current version may have limited diversity

License

This model is released under the flux-1-dev-non-commercial-license, available for non-commercial use only.

This release brings new possibilities to the FLUX ecosystem and is expected to inspire more interesting applications from the community.