Kuaishou Launches ReCamMaster Monocular Video Reframing Technology
Kuaishou Technology recently released a new video processing technology called ReCamMaster. This technology enables users to “re-shoot” existing video material by controlling the camera perspective and motion path, generating entirely new dynamic perspective effects without the need for additional shooting equipment or complex post-production.
Technical Features
ReCamMaster allows users to:
- Recreate videos with new camera trajectories from a single video source
- Provide various preset camera motion modes, including panning, tilting, zooming, and arc movements
- Maintain consistency of scene content and actions in the video
- Be applicable in various scenarios, such as video stabilization, 4D reconstruction, and data augmentation
Application Scenarios
According to the project page, ReCamMaster can be applied in multiple fields:
- Video Stabilization: Capable of converting shaky handheld footage into smooth and stable visuals
- Autonomous Driving: Providing data augmentation for autonomous driving systems, creating multi-angle observation data
- Robot Vision: Offering multi-angle observation data for AI robots
- Video Creation: Providing creators with new video editing and creation tools
Technical Principles
ReCamMaster is based on a pre-trained text-to-video diffusion model. By designing a sophisticated video conditioning mechanism, it can generate videos according to user-specified new camera trajectories while maintaining the original video content. The technical team also utilized Unreal Engine 5 to build a large-scale multi-camera synchronized video dataset for training, enabling the model to better adapt to real-world shooting characteristics.
User Experience
Kuaishou Technology is inviting users to experience the ReCamMaster technology. Users can upload their videos and choose from the following ten basic camera trajectories:
- Pan right
- Pan left
- Tilt up
- Tilt down
- Zoom in (push)
- Zoom out (pull)
- Move up (with rotation)
- Move down (with rotation)
- Arc left (with rotation)
- Arc right (with rotation)
Related Links
This technology is similar to recently popular video generation and editing technologies, such as TrajectoryCrafter, but based on the demonstration results, ReCamMaster seems to provide higher quality outcomes. Currently, this technology is not fully open-source, but the team offers trial channels.