Spark-H3: Block Sparse Attention Speeds Up MiniMax-H3 in ComfyUI

From the project repository: the same prompt, seed and schedule rendered with dense attention, Spark-H3 at 10% density and Sol-H3.

What Spark-Attn changes

Block sparse attention splits a video attention pass into two branches: an exact branch that computes the top-ranked key/value blocks in full, and a compressed branch that covers everything else cheaply. Spark-Attn reworks both halves, and Spark-H3 is its MiniMax-H3 implementation. The technical blog published on 2026-09-23 and was last updated on 2026-10-08.

• Spark-Reblock replaces fixed spatio-temporal tiles with an adaptive partition. Two queries count as similar when they produce similar attention-score patterns over the keys, measured with a Mahalanobis cosine distance induced by the opposite side's second moment. A hierarchical partitioner keeps splitting groups until every leaf holds the target block size, in O(N log N) time, so the block layout follows the input, layer and head instead of a fixed neighbourhood.
• Spark-Reweight replaces the mean-pooled key/value summary of the compressed branch with weighted summaries plus a log-mass bias. Mean pooling underestimates attention mass because attention exponentiates logits before normalising: it substitutes exp(E[x]) for E[exp(x]). The bias restores the missing mass, and scoring uses the mean of the video queries as one anchor per head so the extra cost stays linear.

Both parts are measured on their own, at a 10% Top-K budget:

And for the compressed branch:

Mean pooling produces exp(E[x]), in blue, below the expected exponential E[exp(x)], in orange. The green point is the effective pooled logit after the log-mass bias, whose exponential matches the expected exponential.

Benchmarks

The head-to-head numbers use VBench prompts at 1344x768, 240 frames at 24 fps, and a 19-step MiniMax-H3 schedule on a single RTX PRO 6000 Blackwell. Every sparse method here shares a 20% dense warmup, keeps the first Transformer layer dense throughout, and is compared against dense outputs from the same prompts and seeds.

At 10% density Spark-H3 gains 2.94 dB of PSNR over Sol-H3 while also running slightly faster, and raising the density trades speed back for fidelity monotonically. Spark-H3-Lite is a cheaper variant that halves the reblocking latency for a small fidelity cost at 20% and 30%. On the VBench dimensions, every Spark-H3 variant stays within about 0.3 points of dense on subject and background consistency, motion smoothness, imaging quality and aesthetic quality.

The ComfyUI nodes

The package adds two attention nodes for ComfyUI's own MiniMax-H3 implementation: MiniMax H3 Spark Attention (SM89) and MiniMax H3 Spark Attention (SM120). One goes on the MODEL wire between the model loader and BasicGuider, and only one attention patch should be applied per model. Everything else in the graph stays untouched, so it stacks with the community LoRAs the examples use.

The installer picks the matching backend for the ComfyUI you have: comfy-kitchen 0.2.36 on 0.38.x and 0.2.37 on 0.39.x. Release v0.1.2 added verified Windows packages and refreshed the bundled workflows and install docs; CUDA 12.8 is only reachable through the source-build path. Model files still come from the Comfy-Org MiniMax-H3 repository, and the ComfyUI guide links a Chinese install guide as well.

End to end in ComfyUI, on the same RTX PRO 6000 Blackwell, for a 1344x768 clip of 14.4 seconds:

Dense, Fidelity and Speed attention in one clip, generated with the native MiniMax-H3 workflow in ComfyUI. The prompt comes from Video DeltaNet's prompt set.

Workflows

Five example workflows ship with the release, and each fixes topk_mode=topk_ratio and reblock_layout=q_reuse_k. The native, LightX2V, Larryvrh and DMAD examples use topk_ratio=0.2; the LBH two-pass example uses 0.1 to match the Diffusers h3_lbh path.

The package never installs the extra LoRAs silently: they stay under models/loras and are your call. A conversion script for the Larryvrh weights keeps 208 backbone LoRAs and projects 51 AdaLN updates onto the pruned model's eight-dimensional timestep curve.

A minimalist product-ad prompt rendered with the Spark-H3 backend, from the sample set in the project repository.

Availability

Spark-H3 lives at github.com/zechengtang/Spark-H3, with the code-only mirror at Aazeus/Spark-H3 and the write-up at zechengtang.github.io/Spark-H3. It is an inference backend rather than a checkpoint, so no model weights are redistributed and Spark-H3 can be combined with existing H3 LoRAs. The project roadmap lists SM120 validation as in progress, a standalone release without the comfy-kitchen dependency and kernels for SM90 and SM100 as planned, and the technical report as not yet published. The first ComfyUI release is also young, so expect the packaging and the routing details to keep moving.

If you have already tried Veda sparse attention on MiniMax-H3, the blog includes a discussion of how Spark-Attn, Sol-Attn, Veda and plain block sparse attention differ: Veda learns a scorer for Top-K selection, Sol-Attn uses a content-adaptive threshold, and Spark-Attn changes the block layout and the treatment of unselected blocks.