SCoPE: Camera Control for Wan2.2 Video Generation

SCoPE (HuggingFace | GitHub | Project Page) is a Tencent ARC research release that adds camera trajectory control to the Wan2.2-I2V-A14B video diffusion transformer by encoding each video token's camera ray as a positional coordinate.

Overview

SCoPE (Sightline-Coordinate Positional Encoding) treats the camera ray of each video token as a second positional coordinate — a property of the coordinate system rather than an added control module. Given a first frame, a text prompt, and a camera trajectory, it generates a video that follows the requested camera motion while preserving the original image-to-video prior.

SCoPE overview: camera trajectories drive the generated video (source: model card)

The retrofit keeps RoPE bit-exact, starts from the unchanged pretrained DiT, and adds under 0.1% new parameters. A Normalize-Gate-Inject scheme makes the encoding trainable across both metric and up-to-scale pose sources, so SCoPE can consume poses from different reconstruction pipelines.

How It Works

Camera motion as coordinates. Each video token is tied to its camera ray via Plücker coordinates; no separate control branch is trained.
Robust to heterogeneous pose sources. Per-clip near-depth normalization plus a learned scale gate let SCoPE consume poses from different reconstruction pipelines and scene scales.
Self-contained release. The model repository bundles everything needed for inference — users do not need to download a second Wan2.2 checkpoint (approximately 67 GB total).

Example output: a misty forest scene following a truck-right camera trajectory (source: GitHub)

Demo Video

SCoPE overview demo reel with camera-frustum overlays (video from the official gallery)

Availability

SCoPE is a research release with its own inference pipeline — there is no native ComfyUI support yet. It requires Python 3.11, a CUDA-capable GPU, and the pinned PyTorch 2.9.1 (CUDA 12.8) environment:

git clone https://github.com/TencentARC/SCoPE.git
cd SCoPE
uv sync
source .venv/bin/activate

python inference.py \
  --model_path checkpoints/SCoPE \
  --case omni-misty-forest \
  --trajectory truck_right \
  --output_path outputs/omni-misty-forest.mp4

Camera poses use OpenCV camera-to-world coordinates with shape [81, 3, 4] or [81, 4, 4]; x_fov is the horizontal field of view in radians. Trained on RealEstate10K, DL3DV, PanShot, and OmniWorld.

Key Details

Item	Detail
Base model	Wan2.2-I2V-A14B (self-contained, ~67 GB)
Output	81-frame video following the requested trajectory (e.g. 480x832)
New parameters	Under 0.1% of the pretrained DiT
Training data	RealEstate10K, DL3DV, PanShot, OmniWorld
License	Apache-2.0