Viggle Remake Reshoots Any Video With Two H3 LoRAs

Left: the source clip. Right: the same shot with the pineapple mascot recast, motion, camera and timing untouched.

Viggle has been building in this direction since Viggle-Animate in late August, which replaced a character in a shot from one repainted frame on a heavily distilled H3 ref2va finetune. Remake is what the model card calls the "Viggle-Animate v2" previewed there, and it moves in two ways: it is a LoRA on the stock H3 transformer rather than a finetune, and it edits more than people.

Two modes

Both modes take the source clip plus one to four reference images, called <Picture 1> to <Picture 4> in the prompt. You paint those references yourself, either on top of frames taken from the clip or from scratch, and each one can be pinned to the source frame it was painted from.

• swap replaces people and characters, one or several at once.
• edit does the rest: swap, remove or insert objects, change the background, the weather or the light.

The edit LoRA removing objects from a shot while the camera move and the rest of the plate stay put.

The identity comes from the painted frames, not from a face encoder or a segmentation mask. That is why the pipeline needs no pose estimation or tracking, and also why the model card is blunt that it cannot verify consent: the only place identity is decided is the picture you paint.

What it is, technically

Each mode is a PEFT LoRA, rank 256 and alpha 256, over 302 linear layers of H3's FL2VA transformer, stored in bf16 with four proj_in / proj_out tensors in fp32. Nothing else about the base model changes; the LoRA is a delta applied at load time. The repository also ships the same weights repacked to ComfyUI layer names, merged int8 and GGUF transformers, and a reference sampler in inference/sample.py.

Sound is carried rather than generated. The output keeps the source clip's soundtrack, so a silent source gives a silent result, and the audio encode is padded at the end because ComfyUI's stock audio encode trims a track to whole 800-sample blocks from both ends, shifting it by up to 400 samples against the picture.

The ComfyUI path

ComfyUI runs H3 natively, so the node pack only covers the four things stock nodes do differently. It adds three nodes, tested with ComfyUI 0.42.0:

• Viggle Remake Conditioning, which takes the source frames, the audio, up to four references, the pin frames and the prompt.
• Viggle Remake Keyframe Until, which turns the pinned references into keyframes until a given fraction of the sampled range.
• Viggle Remake Noise, which reproduces the reference sampler's starting noise for a seed.

The graph is otherwise stock: LoraLoaderBypassModelOnly, BasicGuider, KSamplerSelect with euler, ManualSigmas, SamplerCustomAdvanced, two VAE decodes, Create Video at 24 fps.

One setting is worth repeating because it changes the output rather than the speed. The plain LoraLoaderModelOnly merges the LoRA into the bf16 weights, and at bf16 much of the update rounds away: the project measures that only 37 to 73 percent of it survives in the attention and MLP layers, and the merged render on the swap example is 7 percent less sharp (21.2 dB against 24.7 dB). LoraLoaderBypassModelOnly keeps the LoRA as a separate branch for 2 to 3 percent more time per render.

Smaller cards: int8 and GGUF

The bf16 route wants one 80 GB card and about 144 GB of base model. Below that, each mode ships as a single transformer file with the LoRA already merged at fp32 and quantized, replacing both the base diffusion model and the LoRA loader in the graph.

The project's card table, measured with the edit example at 226 frames and two pinned references:

• 12 GB: Q4_K_M or Q8_0 with --reserve-vram 8; int8 runs 124 frames only.
• 16 GB: int8 at 832x480; Q4_K_M or Q6_K with --reserve-vram 6; at 1280x736, Q4_K_M with --reserve-vram 10 and 124 frames.
• 24 GB: int8 or Q4_K_M at 832x480; Q4_K_M or Q8_0 at 1280x736 with --reserve-vram 20.
• 32 to 80 GB: int8 or Q8_0 comfortably at 832x480; at 1280x736 the int8 file runs out of memory at 226 frames on 48 and 80 GB cards, while Q8_0 fits on 24 GB with a reserve.

Quality against the bf16 plus LoRA reference, as PSNR over every frame: int8 scores 27.2 dB on swap and 46.8 dB on edit, Q8_0 26.9 and 49.8, Q6_K 24.6 and 44.6, Q4_K_M 20.6 and 38.3. The swap column is the noisy one: rounding the same bf16 weights a second, slightly different way already moves that render to 29.6 dB, so a few dB there is not a format ranking.

Speed

On one B200 at 832x480 with 226 frames and two pinned references, ComfyUI takes about 95 seconds per render once the models are loaded, roughly 13.5 seconds per step across six steps, against about 58 seconds for sample.py. A first render including loading takes 2.5 to 3 minutes. At 1280x736 a step costs 64 to 72 seconds and a render 510 to 585 seconds. The project only measured B200s and says so; consumer cards will be slower.

Limits

• One render covers a window of 90 to 345 frames (17n+5, 24 fps); the project recommends up to 294, about 12 seconds, and longer videos become windows the MCP server cuts at shot boundaries.
• The six-section H3 prompt is required, and its timestamps have to match the frame count.
• The painted frames decide identity and look; pinned frames also fix composition at that frame.
• Known failures: a non-humanoid character in one long take, small artifacts in fine detail, and shots drifting in time when a multi-shot clip is rendered without pins.

The swap example the ComfyUI workflows ship with: 226 frames, two references pinned at frames 12 and 184.

The edit example: the turbines leave the plate while the pan and the light stay.

Availability

The weights, LoRAs, int8 and GGUF files, node pack and workflows are on Hugging Face, with COMFYUI.md, PROMPTING.md and MODIFICATIONS.md alongside. The repository is at about 210 GB in full, so the ComfyUI guide downloads only the comfyui/ and examples/inputs/ subsets. It was published on November 4, 2026 and announced by @viggleai the same day; the card carries no benchmark against other editing models, so the numbers above are the project's own.