Iris-3B Turbo Runs on CPU and Apple Silicon Without a VAE

Iris-3B is a 3B-parameter diffusion transformer from Speridlabs that denoises RGB pixels directly. There is no VAE and no latent space: the last blocks of the network turn each 16×16 patch back into pixels, so nothing gets decoded at the end (model card | code | paper).

The reference implementation asks for an NVIDIA GPU with CUDA. Two independent ports now run the same weights on hardware most people already own: FastSDCPU added the 4-step Turbo checkpoint to its OpenVINO path on October 18, and Iris-3B Metal runs it on Apple Silicon with custom Metal kernels and a native macOS app.

The ancient city of Petra at morning light, generated by Iris-3B at its default 100-step setting. Source: speridlabs/iris-3b

Inside the model

The architecture is a fairly small diffusion transformer: 8 dual-stream and 16 single-stream blocks, followed by a 4-block pixel head that maps each 16×16 patch of the input back to pixels. Text comes from a frozen Qwen3-VL-4B-Instruct encoder. Training is rectified flow, from scratch at 256, then 512, then 1024 px, with supervised fine-tuning at 1024 px for 665K steps in total.

Skipping the VAE removes the encode and decode stages, but the authors are careful to say that pixel space is not automatically cheaper. The transformer itself carries a heavier compute cost than a latent model of the same parameter count, which is why the CPU numbers below are measured in minutes rather than seconds.

The same backbone is also fine-tuned, with no architectural change, into two dense tasks: monocular relative depth and 4× image restoration. Both run the whole model in a single forward pass with an empty prompt, so they need no text encoder at all.

Turbo: four steps, no CFG

Iris-3B Turbo ships in the turbo/ folder of the same repository and is a full fine-tune of all 3B parameters distilled with Trajectory Distribution Matching. Classifier-free guidance is baked into the weights, so each step is a single forward pass; the model card measures about 0.7 s for a 1024×1024 image on one H100 at batch 4.

Sampling is fixed to a 4-step schedule (σ = 1, 0.938, 0.833, 0.625) with deterministic flow-Euler steps, so --steps, --cfg-scale and negative prompts no longer apply. Turbo was distilled across seven aspect buckets around one megapixel, including 1024×1024, 752×1344, 896×1344 and 896×1200 and their transposes.

An Iris-3B Turbo result, generated on an H100 with seed 6. Source: speridlabs/iris-3b

Turbo is slightly ahead of the base model on prompt following and behind it on long rendered text and on OneIG-Bench, where the gap comes mostly from text and output diversity. The model card also lists grain on flat vector-style backgrounds and less variation between seeds as known trade-offs of the 4-step schedule.

On CPU, through OpenVINO int8

FastSDCPU shipped Iris-3B Turbo support in release v1.0.0-beta.600 on October 18, as an int8 OpenVINO conversion of the transformer and the Qwen3-VL encoder (weights).

The integration is text to image only. Guidance scale, LoRA and ControlNet have no effect on the Iris path, and the README notes it was tested on CPU and integrated GPUs.

On Apple Silicon, through Metal

Iris-3B Metal is an independent MPS and Metal engine with its own kernels, plus a macOS app, a CLI, a Python API and a local HTTP API. Version 0.5.0 runs the official weights with no fine-tuning of its own (repository).

The native macOS app, with an image generated by Iris-3B. Source: TKY-27/iris-3b-metal

On a base M5 MacBook Pro with 24 GiB, the README reports 12.07 s warm for Turbo at 4 steps, 38.38 s for the original model at 10 steps and 76.04 s at 20 steps, with peak process memory around 10 GiB. Machines with 16 GB have not been tested.

Three prompts at 10, 20 and 100 steps with the same seed: composition holds, fine detail keeps changing. Source: TKY-27/iris-3b-metal

Depth and restoration

Both dense tasks reuse the generative prior instead of training a separate vision backbone. On the Marigold V2 protocol, zero-shot across NYUv2, KITTI, ETH3D, ScanNet and DIODE, the depth model reports AbsRel 0.071 and δ1 0.946; depth is relative and affine-invariant, not metric. The restoration and upscaling model reports LPIPS 0.292 and PSNR 20.68 on DIV2K validation at 4×, and tiles large outputs into overlapping 1024×1024 windows.

Availability

ComfyUI has no native loader for Iris-3B yet. Text to image runs through the reference Python scripts, the two ports above, or the Turbo demo Space. Three community node packs exist for ComfyUI, ScryptHunter/ComfyUI-iris, RealRebelAI/ComfyUI-Iris-3B and bani4kaskashka/Iris-3B-Comfy-Nodes, all published in early October and still small.