AlayaWorld: An Open Source Interactive World Model With Real-Time Camera Control and Prompt Switching
Alaya Lab released AlayaWorld, a full-stack open source framework for building interactive generative worlds. It features long-horizon video consistency beyond one minute, real-time camera control, and prompt-driven interaction.
On July 8, 2026, Alaya Lab (affiliated with Shanda AI Research) released AlayaWorld, a full-stack open source framework for building interactive generative worlds. Instead of the traditional game engine approach where every object must be manually authored, AlayaWorld generates entire playable worlds on the fly using an autoregressive world model.
| Project page | alaya-lab.github.io/AlayaWorld |
| Demo video | YouTube |
| Technical report | arXiv 2607.06291 |
| Code & weights | Coming soon |
| License | Apache 2.0 |
AlayaWorld generates playable, interactive worlds across multiple art styles, from realistic scenes to stylized game environments.
Four Core Properties
AlayaWorld is built around four design principles: interaction, consistency, stability, and runtime.
Interaction
Two control channels enable real-time user interaction:
- A rendered 3D cache with lightweight AdaLN camera modulation for grounded, trajectory-aware navigation — users can freely move the camera in any direction
- Chunk-level prompt switching that introduces new events mid-generation, allowing users to trigger actions like combat, spell casting, or monster summoning
Consistency
To keep the world recognizable as you explore, AlayaWorld uses two complementary memory systems:
- An explicit 3D cache reprojected to the queried view for spatial recall
- A compressed frame-history embedding for temporal continuity
This means revisited places stay recognizable, solving a key problem that plagues most video generation models.
Stability
Long-horizon video generation is notoriously unstable — errors compound over time. AlayaWorld addresses this by training on drifted histories and maintaining an error bank that re-injects accumulated artifacts back into both memory and target, preventing errors from compounding over minute-long rollouts.
Runtime
The system achieves real-time interaction through few-step DMD distillation and short temporal chunks. Prompt switching happens at chunk boundaries to minimize both visual and semantic latency.
Style Flexibility
AlayaWorld supports multiple art styles for the same scene. The project page demonstrates a Jiangnan (south-of-Yangtze) scene rendered in:
- Realistic — photorealistic rendering
- Oil painting — expressive brushwork style
- Ink wash — traditional Chinese ink painting
- Cyberpunk — neon-lit futuristic aesthetic
- Zelda-style — cel-shaded game aesthetic
Users can switch between styles or even trigger different styles within the same world session.
Beyond Gaming
While AlayaWorld's interactive capabilities shine in game-like environments — users can navigate freely, fight enemies, cast spells, and summon monsters — the framework is designed to generalize beyond gaming. Trained on both gameplay recordings and real-world videos, it can capture diverse visual appearances and physical dynamics, opening possibilities for embodied intelligence and synthetic environment training.
Team and Roadmap
The project is led by Kaipeng Zhang (Core Lead) and Chuanhao Li (Lead) from Alaya Lab. The full technical report is available on arXiv.
The release roadmap includes:
- Inference code
- Pretrained weights
- Training code
- Training data (partial)
All components are planned for release, making AlayaWorld a full-stack open source framework — from data preparation and model architecture to training, inference acceleration, and deployment.