MiniMax H3 PTD LoRAs Cut Ref2VA to 8 or 4 Steps

PTD stands for Parametric Trajectory Distillation, published as arXiv 2610.11498 by Lan Feng and co-authors from EPFL, NVIDIA and Stanford, with a project page whose film is generated entirely on MiniMax H3. The paper distills the 33B joint audio-video model, sampled in 50 steps, into a four or six step student using nothing but a LoRA and trajectory supervision.

What these two repositories add is the Ref2VA half of that story: reference images in, video plus audio out, at 8 or 4 transformer evaluations.

How the adapters work

The team's earlier PDD release (MiniMax-H3-Acc-LoRAs) replaced the base model's single output head with a bank of per-interval heads. PTD keeps that bank and trains one independent low-rank adapter per inference step, a "per-hop" LoRA on the diffusion backbone, so every hop selects its own adapter and fuses its own head. An 8-hop checkpoint therefore runs in 8 transformer evaluations and a 4-hop checkpoint in 4.

Each checkpoint carries its own configuration in the safetensors metadata (ptd_num_hops, pdd_num_steps=32, pdd_block_size, lora_rank, lora_alpha, lora_targets). The provided apply_ptd_lora reads that metadata and derives the step count on its own, which is why no step slider exists in the ComfyUI nodes: the sigma grid is a property of the checkpoint.

What the demos show

The PTD card reuses three cases from the MiniMax-H3-Turbo examples (poolside fight, warehouse confrontation, motorcycle out of a monitor) and lines them up against the base Ref2VA model, the 4-step Turbo LoRA and the 8-step PDD release. Reference images, prompts and a predict_ref2v.py script ship in the repository, so each column is reproducible.

The RefAsymOPD card goes further and publishes five new cases at 768×1344, 124 frames, 24 fps with audio, each row comparing the 40-step base as a quality ceiling, the 8-step PDD release, an 8-hop plain PTD pass and the 8-hop RefAsymOPD pass. The claim attached to it is narrow: better background plausibility, sharpness and realism than plain PTD at the same step count, with no extra cost, because the checkpoint is structurally identical to a plain PTD LoRA and swaps in without code changes.

One frame from each of the four columns of demo case sample-0, taken from the same point of the clip: base at 40 steps, PDD 8-step, PTD 8-step, and PTD + RefAsymOPD 8-step.

The full 8-step RefAsymOPD clip for sample-0, with its generated audio. The cover image is a two-panel frame from a different case (sample-2).

Running it in ComfyUI

Both repositories ship the same custom node pack, comfyui_node/ComfyUI-MiniMaxH3-PTD/. It does not reimplement PTD sampling itself: it reuses the hook machinery of the upstream ComfyUI-PTD nodes and replaces only the loader, the hop grid and the output heads. That split exists because upstream accepts 4 and 6 hop checkpoints and expects per-hop deltas on the base output head, while these adapters are 4 and 8 hops and were trained on top of PDD's per-interval head bank.

Setup, as documented by the pack:

1. Install upstream ComfyUI-PTD (branch release/comfyui-inference) into custom_nodes/ and pip install -e it. Its ptd_comfy module imports a sibling ptd package, so a bare clone fails at import.
2. Copy ComfyUI-MiniMaxH3-PTD from the repository into custom_nodes/.
3. Put the adapter checkpoints in ComfyUI/models/ptd/.
4. Convert the base yourself: these adapters need the original, unpruned transformer_ref/ from MiniMaxAI/MiniMax-H3, converted with scripts/convert_comfy_base.py. Pruned, Turbo, PDD and quantized bases are explicitly not valid.
5. Start ComfyUI with --disable-dynamic-vram.

The text encoder (qwen3vl_32b_minimax_h3_bf16) and the video and audio VAEs come from Comfy-Org/MiniMax-H3. With the base included, the full set of weights is around 140 GB.

The graph is short: Load Diffusion Model, then Load MiniMax-H3 PTD Adapters, then the PTD Sampler and PTD Sigmas nodes feeding SamplerCustomAdvanced, conditioned through MiniMax H3 Reference to Video, then separate the AV latent and decode video and audio. An API-format example workflow ships in the repository, one per release, and the README warns that its reference images have to be copied from asset/ into ComfyUI/input/ first.

The authors report an end-to-end validation through ComfyUI's HTTP queue on both PTD checkpoints: exactly 8 and 4 transformer evaluations at the checkpoint knots, 1344×768, 243 frames, 24 fps and 32 kHz stereo audio, matching the Diffusers script in scene, identity and audio structure. Same seeds do not reproduce the Diffusers output bit for bit, because the two front ends draw initial noise differently. The tested stack was ComfyUI 5cc4b87, ComfyUI-PTD 43ae22c and PyTorch 2.13.0+cu130 on a single H20-3e.

Community workarounds

Because the adapters require the unpruned 140 GB base, the first community response was to make them fit smaller setups. A day after the PTD release, a compact loader (ComfyUI-PTD-Compact, created October 22) added support for pruned and quantized bases along with int8 versions of the 4-step adapter, and Banodoco users have been posting first stills and clips from those conversions. The step-8 int8 build runs at roughly 14 seconds per still at 4 MP on that author's card.

Availability

Both adapter sets are on Hugging Face: MiniMax-H3-PTD-LoRAs for the 8-step and 4-step pair, and MiniMax-H3-PTD-RefAsymOPD-LoRAs for the RefAsymOPD variant. Inference outside ComfyUI runs through the MiniMax-H3 Diffusers modular pipeline with diffusers >= 0.40.0 and the ref2va workflow.