Anima Reference Adapter: One Image, New Scene in ComfyUI

Tag-prompted anime models only draw who you ask for if the prompt spells out every trait, and only for characters the model already knows. A reference adapter takes that job away from the text: point it at one image, and the prompt is free to describe pose, framing and location.

Most reference conditioning borrowed from video models does not make that trade. ComfyUI-Cosmos-Reference appends a reference latent as an extra frame of the clip, which works well for editing but also copies the reference's pose and layout, because the video prior reads that frame as the previous shot. Anima Reference Adapter asks for the opposite: keep who is drawn and how, drop where and in what pose.

How the reference is attached

Anima runs on the Cosmos-Predict2 MiniTrainDIT: 28 blocks, width 2048, 16 heads, 2×2 spatial patches over a 16-channel Wan 2.1 VAE latent, with a Qwen3-0.6B text encoder and flow matching at shift 3. That gives it 3D rotary position embeddings over (t, h, w), which is the lever the adapter uses.

The reference image is letterboxed into the output resolution, VAE-encoded once, and appended as a second latent frame. Three changes separate it from a plain video-style reference:

• Clean reference frame. The reference frame is fed at timestep 0, the target frame at σ. Nothing is noised on the reference side.
• Offset RoPE. The reference frame's height coordinates are shifted past the target grid (by one target height, plus a small jitter during training), so no pixel-to-pixel correspondence survives and the model cannot continue the reference as a shot.
• A trained reader. A rank-32 LoRA with α = 32 on every block's self-attention, cross-attention and both MLP layers (280 modules, 45.9M parameters) learns to read that displaced frame. It is applied through forward hooks and only during model calls that carry a reference, so a loaded base model can still run a reference-free pass.

Guidance is split as well. With the reference in both CFG branches, only the text gets amplified; the guider instead uses separate text and reference scales, so ref_guidance can move the result towards the reference without touching cfg. High reference guidance over-saturates, so the node applies either adaptive projected guidance or CFG rescale on top.

What the held-out characters show

Training data is synthetic and made by Anima itself: 600 groups of one character in one style, six renders each with independently sampled framing, camera, pose, expression, background and lighting, for 3,600 images across 300 characters and 427 artists on a single RTX PRO 6000 Max-Q in 4.25 hours. Twelve characters were held out entirely; the remaining 576 groups produced 17,280 ordered reference-target pairs, 70% of which dropped all appearance and outfit tags from the target prompt.

Held-out characters, never seen in training. Each row: the reference; the adapter with a scene-only prompt; the same prompt without the adapter; and the base model asked for the character by name with full appearance tags.

Across the 18 evaluation cases the adapter reproduces hair colour and styling, eye colour, outfit and signature accessories such as horns and a tail, a halo, butterfly wings or a maid headdress, while the same scene-only prompt on the bare base model returns a generic character. Pose, framing and background followed the prompt in every case, with no pose copying observed.

Two ordinary references show the same split outside the held-out set:

Reference A, the only image given to the adapter	cowboy shot, looking at viewer, smile, beach, ocean, sunset

The prompt carries no character name, artist tag or appearance tags. Recommended settings are Euler at 28 to 30 steps with simple, text CFG 4.5, reference guidance 4.5 to 6 and APG, and bf16 compute, since Anima overflows in fp16.

Style is the weaker half

Original characters drawn by the base model with an artist tag that never appeared in training. Each row: reference; adapter with a scene-only prompt; base model with artist and description tags.

Character design transfers well: gradient twintails, star hair ornaments, a neckerchief, silver hair with a red scarf and gloves. The artist's painterly rendering, soft lighting and muted palette do not come across at reference guidance 1, and the outputs fall back to a crisper default Anima look.

Crops at reference guidance 1, 2.5, 4.5 and 6 next to the reference, with the base model plus full tags on the right.

Raising reference guidance brings back hair texture and highlights, eye rendering, the cool palette and small accessories such as a bandage or heterochromia that guidance 1 drops. Past about 4.5 colours over-saturate and existing drift gets amplified, which is what the APG or rescale correction is for. The painterly rendering still does not appear, which the report attributes to the LoRA having learned little of it rather than to under-amplification. The author is generating a larger set of 2,000 characters in three artist styles and plans separate character and style references.

The nodes

ComfyUI core supplies the rest: UNETLoader for anima_baseV10.safetensors, CLIPLoader for the Qwen3-0.6B encoder, VAELoader for the Qwen-Image VAE. The basic graph runs in 19 nodes on one SamplerCustomAdvanced; the second workflow adds Anima Safe PAG. With ref_guidance at 1 the reference sits in both CFG branches and each step costs what a normal CFG step costs; above 1 every step runs the model three times, or about 1.5× a normal CFG step. The example workflows also use Patch Sage Attention from ComfyUI-KJNodes, which can be bypassed.

Limitations

The pilot was trained on 300 characters for 2,000 steps at rank 32, and the author lists what that costs: identity transfers far better than style, strongly painterly styles barely transfer, small patterns, scars, earrings and unusual skin tones are often lost, large garments can drift in colour, and the reference character occasionally reappears as an object in the scene. Training pairs are all Anima renders, so the adapter inherits the base model's biases, and evaluation so far is qualitative with no identity or style similarity metrics.

Availability

The adapter weights are on Hugging Face as anima_ref_adapter_pilot_r32.safetensors (about 92 MB) and go into ComfyUI/models/loras/. The node pack installs with git clone https://github.com/ElectronGirl/ComfyUI-Anima-Ref-Adapter.git inside ComfyUI/custom_nodes, and needs Anima base v1.0 together with its Qwen3-0.6B text encoder and Qwen-Image VAE. The adapter was trained on the base model, so the Anima turbo LoRA can still be loaded in front of the adapter loader using its usual steps and CFG.