Bernini-Diffusers-v2: ByteDance Open-Sources Full Video Pipeline

Overview

Bernini is ByteDance's unified framework for video generation and editing that combines an MLLM-based semantic planner with a DiT-based renderer. Instead of generating frames directly, it first decomposes complex instructions into explicit semantic plans, then renders the result with a Wan2.2-based diffusion decoder. This "latent semantic planning" gives it strong instruction following for complex generation and editing requests.

Bernini-Diffusers-v2 packages the entire pipeline in one self-contained diffusers directory: a Qwen2.5-VL planner, the Bernini planning weights, and the Wan2.2 diffusion components (text encoder, VAE, scheduler, dual transformer configs). Compared with the renderer-only Bernini-R releases, it is recommended when you need multi-step semantic planning and better handling of complex requests. Compared with the first Bernini-Diffusers release (June 2026), v2 uses a training recipe that warms up the connector for thousands of steps before co-training, improving reference-guided video editing and OpenS2V performance.

The Bernini framework: an MLLM-based semantic planner decomposes instructions into plans, which the Wan2.2 DiT renderer turns into video. Source: Bernini project page

Supported Tasks

Bernini-Diffusers-v2 covers six tasks through ready-to-run launchers in the Bernini repository:

Benchmark

On video editing, Bernini reaches the first tier among leading closed-source commercial models in ByteDance's internal arena evaluation based on blind human pairwise comparisons.

Package Layout

The release is a self-contained diffusers-format directory. Pass the downloaded directory directly to --config:

Bernini-Diffusers-v2/
  bernini/          # Bernini planning checkpoint
  mllm/             # Qwen2.5-VL planner assets
  scheduler/        # base diffusion modules
  t5_text_encoder/
  t5_tokenizer/
  vae/
  config.json
  transformer_config.json
  transformer_2_config.json   # Wan2.2 diffusion decoder components

ComfyUI Support

The Bernini-R renderer has official ComfyUI support from Comfy-Org, with a dedicated tutorial at docs.comfy.org covering text-to-video, image-to-video, and video editing workflows. The diffusers-format directory from this release can also be loaded with ComfyUI's standard Load Diffusion Model node.

For the full semantic-planning pipeline (planner + renderer), the official Bernini repository provides Python inference code, including single-GPU and multi-GPU (torchrun --ulysses 8) Gradio demos and per-task run scripts under scripts/bernini_v2/ (run_t2i.sh, run_i2i.sh, run_t2v.sh, run_v2v.sh, run_rv2v.sh, run_r2v.sh).

Availability

Download the model with the hf CLI:

pip install -U "huggingface_hub"
hf download ByteDance/Bernini-Diffusers-v2 --local-dir pretrained_models/Bernini-Diffusers-v2

Then run inference by passing the directory as --config:

git clone https://github.com/bytedance/Bernini.git bernini && cd bernini
pip install -r requirements.txt
python infer_single_gpu.py --config pretrained_models/Bernini-Diffusers-v2 \
    --case assets/testcases/i2i/i2i.json --num_frames 1

The paper is available at arXiv:2605.22344.