InstantX Releases FLUX.1-dev IP-Adapter Model
InstantX team officially open-sourced the FLUX.1-dev IP-Adapter model on November 22, 2024. This is an IP-Adapter implementation based on FLUX.1-dev, bringing powerful image reference capabilities to the FLUX model.
FLUX.1-dev IP-Adapter Key Features
- Uses google/siglip-so400m-patch14-384 as image encoder
- Implements simple dual-layer linear MLPProjModel for projection
- Image token count set to 128
- Trained on 10 million open-source datasets with batch size 128 for 80,000 steps
Technical Implementation
The model adds new layers in 38 single-layer blocks and 19 dual-layer blocks. It utilizes google/siglip-so400m-patch14-384 for image encoding, chosen primarily for its outstanding performance.
Usage Methods
Currently supports two main usage methods:
- Direct calling through Python code
- Using the ComfyUI-IPAdapter-Flux plugin in ComfyUI
ComfyUI Workflow
InstantX provides example workflow files for immediate use:
Online Experience
Users can experience FLUX.1-dev-IP-Adapter through the following platforms:
Open Source Resources
Related resources are now open-sourced:
- Model Download: FLUX.1-dev-IP-Adapter
- ComfyUI Plugin: ComfyUI-IPAdapter-Flux
Usage Limitations
The model currently has some limitations:
- Not suitable for fine-grained style transfer or character consistency
- Trade-offs between content leakage and style transfer
- Current version may have limited diversity
License
This model is released under the flux-1-dev-non-commercial-license, available for non-commercial use only.
This release brings new possibilities to the FLUX ecosystem and is expected to inspire more interesting applications from the community.