H3 Motion Keyframes a Subject and Lets MiniMax H3 Animate It

ComfyUI Wikinews

H3 Motion is a local app that drives ComfyUI: click a subject out of a photo with SAM 3.1, drag it along a keyframed path, and MiniMax H3 renders the shot with sound.

<strong>H3 Motion</strong> is a local React app that drives a self-hosted ComfyUI to animate one thing in a still or a video. Pick a subject out of a photo with SAM 3.1, drag it along a keyframed path, and MiniMax H3 renders the shot with sound. On video, draw a mask, animate the mask, and H3 redraws only what is inside it. The author, PxTicks, built it for Comfy's open-source Higgsfield competition and already ships Vlo, the timeline editor this site covered in September.
The Move phase: each subject keeps its own path, with diamond keys on a shared timeline

Move phase. Every selected subject gets its own coloured path and keys, and the clip length snaps to H3's frame grid.

This is not another H3 Motion Context. That node pack chains finished clips so motion continues across a cut; H3 Motion works on one clip and moves a subject inside it.

Photo: select, cut out, drag a path

1. Select. Describe the subject by name and SAM 3.1 finds it. Clicks refine the mask: add or remove points until only the thing you want is included. 2. Isolate. Qwen3-VL describes the confirmed subject, Qwen Image 2.1 removes it, and the app shows the clean background. You can retry with a new seed or rewrite the removal instruction. 3. Move. Drag the subject to record a path, or scrub the timeline and drag to place keys one by one. Keys also set size, so an object can grow, shrink, or flip past its edge to fake a turn. 4. Generate. The app renders the keyframed frames and hands them to MiniMax H3 with Time-to-Move, which returns a clip with sound.

Clip length runs from 3.0 to 15.1 seconds in 17-frame steps, the grid H3 samples on, and the first frame is always your original photo. Paths survive a background retry, so a subject does not have to be redrawn after a failed isolation.

The Select phase, with a subject picked out by SAM 3.1 and refined by clicks

Select phase. The prompt finds a candidate; points decide the final mask.

Video: animate the mask, not the frame

Draw a rectangle or ellipse over part of a clip. The mask can be animated by hand with the same path tools, or told to follow an object: SAM 3.1 segments it on the first frame and the core SAM3_VideoTrack node propagates it, with the app rejecting frames whose area or position jumps. Describe what should appear, and optionally supply a reference image, generated from a description or made by editing the current frame inside the mask. H3's ref2va model regenerates the masked region in one sampling pass and the app composites the result back with a feathered edge, keeping, referencing, or replacing the original audio.

The demo clip shipped in the repo's web assets (1280×720, 4 seconds).

Running it

The app is source only: clone it, npm install, npm run dev, open http://127.0.0.1:5180. It connects to a ComfyUI you already run on port 8188 and starts the API proxy itself; it never launches ComfyUI or downloads models. A second terminal runs npm run check-env to confirm every node and model file is present.

You need ComfyUI (the README was tested at v0.41.0-18), ComfyUI-vlo 0.3.0 or later for the vloTimeToMove, vloSAM3RefineMask and vloMaskToLatentMask nodes, Node 22.12 or newer, and uv. Both H3 workflows use Comfy Kitchen INT8 attention, so an NVIDIA or AMD GPU is required. The README lists exact hf download lines for the eleven files: SAM 3.1, Qwen Image 2.1 and its Qwen3-VL encoder and VAE, the H3 fl2va and ref2va diffusion models, the H3 text encoder and both VAEs, plus the Pika Flex LoRAs.

Limits

  • Native Windows is untested; WSL is the tested path. A hosted Comfy API deployment is described but also untested.
  • Photos and videos must sit between 1:4 and 4:1, since anything wider than 4:1 cannot be delivered at 768p.
  • No tagged release yet, and the last commit was on November 3.

Availability

Comments

Sign in with GitHub to join the discussion.

Loading comments…
H3 Motion Keyframes a Subject and Lets MiniMax H3 Animate It | ComfyUI Wiki