Bernini-Diffusers-v2: ByteDance Open-Sources Full Video Pipeline

Overview

Bernini is ByteDance's unified framework for video generation and editing that combines an MLLM-based semantic planner with a DiT-based renderer. Instead of generating frames directly, it first decomposes complex instructions into explicit semantic plans, then renders the result with a Wan2.2-based diffusion decoder. This "latent semantic planning" gives it strong instruction following for complex generation and editing requests.

Bernini-Diffusers-v2 packages the entire pipeline in one self-contained diffusers directory: a Qwen2.5-VL planner, the Bernini planning weights, and the Wan2.2 diffusion components (text encoder, VAE, scheduler, dual transformer configs). Compared with the renderer-only Bernini-R releases, it is recommended when you need multi-step semantic planning and better handling of complex requests. Compared with the first Bernini-Diffusers release (June 2026), v2 uses a training recipe that warms up the connector for thousands of steps before co-training, improving reference-guided video editing and OpenS2V performance.

The Bernini framework: an MLLM-based semantic planner decomposes instructions into plans, which the Wan2.2 DiT renderer turns into video. Source: Bernini project page

Supported Tasks

Bernini-Diffusers-v2 covers six tasks through ready-to-run launchers in the Bernini repository:

Benchmark

On video editing, Bernini reaches the first tier among leading closed-source commercial models in ByteDance's internal arena evaluation based on blind human pairwise comparisons.

Package Layout

The release is a self-contained diffusers-format directory. Pass the downloaded directory directly to --config:

Bernini-Diffusers-v2/
  bernini/          # Bernini planning checkpoint
  mllm/             # Qwen2.5-VL planner assets
  scheduler/        # base diffusion modules
  t5_text_encoder/
  t5_tokenizer/
  vae/
  config.json
  transformer_config.json
  transformer_2_config.json   # Wan2.2 diffusion decoder components

ComfyUI Support

Bernini-Diffusers-v2 is not yet supported in ComfyUI. The full pipeline is built around the BerniniModel architecture (an MLLM semantic planner plus Bernini planning weights), which ComfyUI does not currently load. The release is a self-contained diffusers-format directory meant for the official Bernini repository inference code: single-GPU and multi-GPU (torchrun --ulysses 8) Gradio demos and per-task run scripts under scripts/bernini_v2/ (run_t2i.sh, run_i2i.sh, run_t2v.sh, run_v2v.sh, run_rv2v.sh, run_r2v.sh).

If you want to run Bernini in ComfyUI today, the Bernini-R renderer has official ComfyUI support from Comfy-Org, with a dedicated tutorial covering text-to-video, image-to-video, and video editing workflows. Bernini-R is the renderer-only variant; it lacks the multi-step semantic planner of the full pipeline.

Availability

Download the model with the hf CLI:

pip install -U "huggingface_hub"
hf download ByteDance/Bernini-Diffusers-v2 --local-dir pretrained_models/Bernini-Diffusers-v2

Then run inference by passing the directory as --config:

git clone https://github.com/bytedance/Bernini.git bernini && cd bernini
pip install -r requirements.txt
python infer_single_gpu.py --config pretrained_models/Bernini-Diffusers-v2 \
    --case assets/testcases/i2i/i2i.json --num_frames 1

The paper is available at arXiv:2605.22344.