SMD: 4-Step Distillation for Images, Video, 3D and MiniMax H3

Outputs from the released SMD students: image generation and editing, 3D assets, and video with synchronized audio.

What SMD changes

SMD starts from plain on-policy velocity regression: the student is trained on its own trajectory instead of on teacher samples. Two additions carry the rest of the method. Pivot is a training-only residual proxy that keeps the local corrections a teacher makes inside a coarse transition apart from the student's own interval-mean prediction, so a 4-step student can express teacher velocity variation it could not represent on its own. The second is a repulsion term that pushes the student away from weaker teacher predictions, which the paper describes as a lightweight way to avoid undesirable modes.

Nothing is added to inference: no online score estimation, no fake-score model, no adversarial discriminator, and the deployed generator keeps its architecture. That is what lets the same method ship as a LoRA on top of an existing checkpoint, or as a full denoiser, in the same release. The paper reports scaling to 33 billion parameters and support for cross-model, multi-teacher and causal-video distillation.

What was released

Two side branches go past single-teacher distillation. FLUX.2 9B is used as a teacher for both Klein 4B and Lens, and SD3.5-Medium comes in six variants, self, clipscore, geneval, ocr, pickscore and multi, where each specialist optimises a different reward.

Every repository in the collection carries weights plus a worked example with its prompt or reference inputs, seed and sampling settings. LoRA packages need the matching base model and revision, and a full transformer still takes its encoders and decoders from the base.

Examples across the release

The six SD3.5-Medium students compared side by side, including the multi-teacher multi variant.

The image students use four sampling steps each, and the model cards show the settings that produced them: a canvas tote bag for FLUX.1-dev, afternoon tea for Qwen-Image, a source and edited pair for Qwen-Image-Edit-2511, and two-teacher comparisons for FLUX.2 Klein 4B and Lens. The video side runs at four steps on Wan2.1 14B and on the causal 1.3B model, and the 3D side ships four-step LoRAs for Hunyuan3D-2.1 Shape and for both TRELLIS.2 stages, with a turntable video for each.

MiniMax H3 in ComfyUI

The four H3 students have their own packages that run on stock ComfyUI nodes, with no custom node pack and no checkout of the SMD training repository:

Each package holds the converted LoRA, a complete workflow and the example inputs. Its download_models.py checks the model files, copies the example inputs and adds the workflow to ComfyUI's default user library, so it shows up under SMD in the Workflows sidebar; the workflow JSON can also be dragged onto the canvas. The reference-conditioned graphs use UNETLoader, LoraLoaderModelOnly, MiniMaxH3SigmaShift, MiniMaxH3ReferenceToVideo and ManualSigmas, so the LoRA does the work and the sampling schedule comes from the package. Switching between 4 and 8 steps means opening the matching package's workflow and LoRA.

A five second H3 reference-to-audio-video take from the 4-step Ref2VA package, generated with the two reference images shipped in the repository.

The packages warn about one trap: substituting H3 weights into ComfyUI's default image template can fail with a missing audio_scale error, so the complete workflow from the package should be used instead.

Training your own student

The training code is in the same repository as the inference examples. It runs on Linux x86_64 with Conda: environments/setup_conda.sh builds an environment with Python 3.12, PyTorch 2.7.1 and CUDA 12.6 wheels, and FlashAttention is compiled from source. A run is a torchrun call against one of the training configs with a data path, for example two GPUs with a Parquet file holding a prompt column for Qwen-Image, and --download-resources fetches what the config needs.

Training builds on NVIDIA FastGen for the training infrastructure and uses model implementations from Hugging Face Diffusers; the H3 export path writes a LoRA adapter with its sampling metadata, so no fused base checkpoint is needed. The repository also ships agent skills for environment setup, training and inference, and for onboarding a new model.

Limits

• The step counts and quality claims are the authors' own: at 4 NFE, SMD is reported to beat strong few-step baselines and to surpass multi-step teachers on several benchmarks. Those numbers come from the paper, not from an independent reproduction here.
• Distilled students are tied to the base revisions and sampling settings in their own package. The H3 students, in particular, are LoRA adapters with cached conditions.
• The training repository targets Linux with Conda and a CUDA 12.6 capable NVIDIA driver; the 3D stages need a second environment.
• Of the released students, the two H3 reference-conditioned packages have the most downloads so far (about 1,500 and 1,200 at the time of writing), while the image and 3D repositories are still in single or double digits.

Availability

Code: zjwfufu/SMD
Project page: zjwfufu.github.io/SMD-Page
Paper: arXiv 2610.18472
Weights: Hugging Face collection
Datasets: zjwfufu/smd-data
H3 packages: T2VA 4-step · T2VA 8-step · Ref2VA 4-step · Ref2VA 8-step
Base model: MiniMax H3