H3-World: Keyboard-Controlled Interactive World Model on MiniMax H3
H3-World turns MiniMax H3 into an interactive world model: a 65.6M LoRA plus an attention patch maps WASD and camera keys to action-controlled video from a single frame.
H3-World generates action-controlled video from a single first frame: character motion on WASD, camera motion on IJKL.
A world model on top of a video model
H3-World is described by its authors from Tencent, the National University of Singapore and The Hong Kong Polytechnic University as the first interactive world model built on MiniMax H3. Instead of generating a fixed clip, it takes an initial frame plus keyboard controls and produces video where the character walks and the camera moves according to your input, with the paper available on arXiv.
The trick is a translation layer between game-style controls and the video model. H3-World converts each keyboard state into one language instruction per future video latent, then uses directed attention routing to bind every instruction to its corresponding latent interval. Character controls use W, A, S, D; camera controls use I, J, K, L, with F for fast camera motion. Presets cover still, forward, back, strafe left/right, tilt up/down and pan left/right, including fast pans.
Small adapter, big backbone
Training uses 8,000 gameplay clips from the ABot-World-Explorer-500h dataset. The result is a rank-32 LoRA of 65.6M parameters, only 0.199% of the 33B MiniMax H3 backbone, released as step-10000.safetensors under Apache 2.0.
| Item | Detail |
|---|---|
| Base | MiniMax H3 33B (FL2VA) |
| Adapter | Rank-32 LoRA, 65.6M params (0.199%) |
| Training data | 8,000 clips from ABot-World-Explorer-500h |
| Controls | WASD character, IJKL camera, F fast camera |
| License | Apache 2.0 (LoRA), H3 license still applies to the base |
How to run it
The LoRA alone is not enough: the authors are explicit that loading the checkpoint through an unmodified MiniMax H3 pipeline will not reproduce the reported behavior. The released checkpoint requires the MiniMax H3 base weights (about 135 GB) plus the official H3-World code repository, which applies a directed-attention patch to a pinned DiffSynth-Studio revision. Inference takes a first frame and a scene prompt, then maps an action preset (or raw key states) to the per-latent instructions, generating 5.2-second clips by default at 832x480.
An interactive demo Space is available at hugging-apps/h3-world-action-demo if you want to see the keyboard-driven control loop before committing to the 135 GB download.
Comments
Sign in with GitHub to join the discussion.