SolarRenderer: Four-Step Structure Rendering on MiniMax H3

SolarRenderer turns a structure video into finished footage on MiniMax H3: a grey box from a game engine, a semantic segmentation map, a set of skeletons, a depth pass, a SMPL body mesh, or a sketch the model never saw in training. Two rank-384 LoRAs do the work, a control LoRA that feeds the structure into H3's own reference pathway and a four-step LoRA that renders the clip in four sampling steps.

The project is from Guian Fang, Shengju Qian, Dongyue Lu, Junchao Huang, Lutao Jiang, Xin Wang, Yiren Song, Li Jiang and Mike Zheng Shou, across NUS, LIGHTSPEED, CUHK(SZ), SLAI and HKUST(GZ). The weights went up on Hugging Face on October 13, 2026, the code repository with the training data and the benchmark followed on October 15, and the paper is arXiv 2610.28630.

A frame from the benchmark's grey box castle case, rendered from the structure video alone plus a first frame and a prompt.

What it takes and what it gives back

The inputs are a structure video at 24 fps with at least 158 frames, an optional first frame in the look you want, and a prompt. The output is 158 frames at 24 fps, landscape 1344x768 or portrait 768x1344, sampled in four steps. Other canvas sizes are resized and cropped. The prompt is not free text: the model was trained on one six-part form (subject definitions, summary, retention analysis, a 350 to 500 word description, soundscape, non-diegetic music), and the repository states plainly that a short free-form sentence works much worse.

The control LoRA carries no condition-type label. Structure frames enter H3's native reference pathway at the positions of the frames they describe, and the prompt names the structure in words, which is how a single adapter covers structural representations it was never trained on. The benchmark lists 16 structure types, 11 of them unseen in training, and the paper's gallery shows the unseen ones holding: pencil storyboards, point clouds, voxels, wireframes, a child's crayon drawing.

The full 158-frame cyberpunk render, re-encoded from the released benchmark output.

Two LoRAs, four steps, and a streaming pair

The control LoRA is rank 384 and injects the structure video frame-aligned. The four-step LoRA is also rank 384 and is fine-tuned from DMAD, the four-step H3 distillation the community has been converting for ComfyUI since early October; here the distillation is retrained with structure guidance so the four-step student keeps the control behaviour. The paper measures that path at 12.3 times faster than 50-step sampling.

Both LoRAs come in a second, block-causal version. The causal pair renders the clip block by block, each block of at most five latent frames seeing the prompt, the first frame, the structure of its last six blocks and the five blocks already rendered before it, never a later block. Every block is four steps and then committed to a key/value cache, so a block can be shown as soon as it is done, and --seconds extends a render to the length of the structure video with the memory of a six-block window.

The pipeline from the project: structure video in, control and four-step LoRAs on a frozen H3 backbone, block-causal variant for streaming.

Running it in ComfyUI

The repository ships its own node pack in comfyui/, cloned into custom_nodes/ the usual way; there is no registry entry, so ComfyUI Manager will not find it. It runs on ComfyUI's own MiniMax H3 support (v0.30.0 or later) with the Comfy-Org/MiniMax-H3 weights, and the LoRAs load as released: the loader maps them onto ComfyUI's weight names, and two are chained, the four-step LoRA first, then the control LoRA.

Three workflows come with it: full quality, low memory, and the long-clip causal graph. The low-memory workflow is the interesting one for local cards. The repository reports it completing with ComfyUI held to 14 GB of GPU memory and peaking at 14.5 GB including the CUDA context, on a 16 GB card such as an RTX 4080 or 4090, with 64 GB of system memory. On a card that holds every model, that path takes about twice as long as the full one.

The pack also serves a page at http://127.0.0.1:8188/solarrenderer: pick an example or drop in a structure video and a first frame, write the prompt, choose the models, and it renders, showing sampler previews as they arrive and playing the result beside its structure. Set --preview-method taesd with taeh3.safetensors in models/vae_approx to watch each step. Only the web capture node needs extra installs (playwright and a Chromium download).

Full quality workflow Low memory workflow (16 GB) Long clip causal workflow

What the benchmark says

The paper scores 208 hard cases and 53 held-out clips with a Gemini pairwise judge, reporting Elo per question, plus an automatic score from DOVER, LAION aesthetic, VideoScore2, flicker, DINOv2, CoTracker3 and edge F1. On the held-out split the comparison includes DISTS, LPIPS, PSNR and SSIM against the real clip.

Ties are real in this table: LynnReal-Omni matches SolarRenderer on the hard split's overall rating and loses on the held-out split. The causal variant is rated only against the bidirectional one, which is why it sits below it. The judge verdicts and 522 renders behind these numbers are released with the benchmark, so the comparison can be re-scored rather than taken on trust.

The paper also claims the causal variant renders 768p in real time. The repository's own Reactor integration, a development preview contributed by the Reactor team, is more conservative: on four B200 GPUs a steady block of 17 frames took about 1.8 to 2.1 seconds, below the native 24 fps timeline, and preparing a new video could take over a minute.

What the community found

On the Banodoco Discord the release got a hands-on pass the day it surfaced. Kijai tested the four-step LoRA against the original DMAD and found that the latent guide has to be used to match the training setup before the result lines up, after which, in his reading, native H3 nodes can carry it: "you do have to use the latent guide to match their training, works better then, we can do it with native nodes". His notes on the outputs were mixed but specific: motion following is accurate to the point of being slightly too literal, and dropping the first frame costs quality.

The prompt form drew the most complaint, with Kijai calling the official example prompt very complicated and RuneX suggesting a system prompt. RuneX also hit a host buffer error running the pack's page, and read most of the pack as replaceable if the model is already loaded natively: "the other ones is just a flash attention node, and a manual sigma node". Alisson, whose TARA patch took the same native-reference route for H3 motion transfer, put the training scale in perspective: 136,000 clips against his own 700.

Reactions came from #minimax_h3_chatter on October 25, starting here.

The released training sources: Magpie, CARLA and UnrealZoo clips, 57,738 of the 136,000 the paper describes.

Limits worth knowing

• The trained clip is 6.6 seconds. Longer videos need the causal pair, and the causal sampler keeps its key/value cache around 80 GB at 1344x768, spilled to system memory when it does not fit.
• The reference path is 135 GB of H3 weights plus 8 GB of LoRAs for the bidirectional pair; the ComfyUI low-memory route is what makes a 16 GB card workable.
• The prompt form is a hard constraint, not a style suggestion. Free text loses the control the LoRA was trained for.
• The Elo table comes from one judge, and the hard split's first frames are made by an image model rather than shot as photographs.
• The Reactor streaming path is a preview, not bitwise identical to native inference, and Docker execution is still unvalidated.

Availability

Weights, causal pair, config and the license terms are on Enderfga/SolarRenderer; code, the ComfyUI node pack, three workflows, the prompt skill and the Reactor integration are in the GitHub repository; the training data and benchmark live in Enderfga/SolarRenderer-Data. There is an online demo on Hugging Face Spaces for a render without an install.