Bernini-R S2V: Speech-Driven Video Generation on ByteDance's Bernini-R
rzgar releases Bernini-R-S2V, a speech-to-video fine-tune that adds Wan2.2 S2V audio-driven lip-sync capabilities to ByteDance's Bernini-R model, with dedicated ComfyUI custom nodes and multiple precision variants.
Community developer rzgar has released Bernini-R-S2V, a speech-to-video (S2V) fine-tune that grafts Wan2.2 S2V audio-driven lip-sync capabilities onto ByteDance's Bernini-R model. The release brings expressive talking-head generation to the Bernini ecosystem, with dedicated ComfyUI custom nodes and support for FP16, FP8, and INT8 precision.
Speech-driven video generation on Bernini-R using audio-driven lip-sync. Source: rzgar/Bernini-R-S2V
What is Bernini-R?
Bernini-R is ByteDance's open-source video generation and editing framework built on the Wan2.2 DiT architecture. It combines an MLLM-based semantic planner (Qwen2.5-VL) with a dual-expert Diffusion Transformer renderer, supporting text-to-video, image-to-video, video-to-video, and multimodal video editing tasks. The model excels at instruction-following for complex generation and editing requests, with official ComfyUI support from Comfy-Org.
Speech-Driven Video on Bernini-R
The Bernini-R-S2V fine-tune adapts the Wan2.2 S2V (speech-to-video) module — originally developed by Wan-AI for audio-driven talking-head generation — and integrates it into the Bernini-R architecture. Users can now drive Bernini-R generated characters with speech audio for natural lip-sync, supporting both image-to-video and video-to-video workflows.
The model is available in multiple precision variants:
| Variant | Size | Notes |
|---|---|---|
| FP16 | ~27 GB | High-noise and low-noise checkpoints, full precision |
| FP8 (scaled) | ~14 GB | Reduced memory with scaled quantization |
| INT8 (ConvRot) | ~13 GB | convrot quantization for aggressive memory savings |
Each noise level comes in two variants: high_noise (more motion/detail) and low_noise (more faithful to source, smoother).
ComfyUI Integration
The HF repo bundles a dedicated ComfyUI-WanBerniniS2V_v2 custom node set as a zip archive, complete with ready-to-use workflow JSONs (Workflow V2, original workflow, audio preparation). A community mirror is also available on GitHub via AIMixer.
BerniniS2V Conditioning V2 node workflow for single/dual speaker lip-sync with masking. Source: rzgar/Bernini-R-S2V
FP8-scaled inference result — speech-driven video on Bernini-R with lip-sync. Source: rzgar/Bernini-R-S2V
Key components of the custom node:
- BerniniS2V Conditioning — handles audio embedding and S2V conditioning for Bernini-R
- Workflow V2 — an updated workflow template for image-to-video S2V generation with masking support, including single and dual-speaker modes
- Audio processing — integrates wav2vec2 audio encoder for speech feature extraction
Banodoco community members have shared working workflows for the node, including setups with additional TTS nodes for text-to-speech-to-video pipelines.
Try It Online
A Hugging Face Space by hugging-apps provides a talking-head demo running on ZeroGPU. Upload a portrait photo and a speech clip to generate a lip-synced talking-head video using 4-step LightX2V distilled sampling for fast inference.
Community Reception
The model has quickly gained traction in the generative AI community, particularly among Bernini-R users who wanted S2V capabilities. Banodoco Discord discussions highlight both enthusiasm for the new capability and ongoing workflow optimization, with users sharing masking techniques and TTS integration approaches. The model currently sits at 7 likes and 158 downloads on Hugging Face.