ComfyUI Adds Core Sparse Attention Node and Comfy Compiler

Block Sparse Attention: one node, three backends

The new Block Sparse Attention node (category advanced/model, marked experimental) lives in comfy_extras/nodes_sparse_attention.py and routes each query block to only a selected subset of key blocks. The relative speed-up grows with sequence length, which is exactly the regime of 10-second H3 clips. A min_tokens guard keeps short sequences dense automatically.

Three selection modes ship in the same node:

• Sol-Attn (adaptive tau): a per-head adaptive threshold in score-distribution sigmas. tau 1.0 keeps roughly 16% of key blocks exact, 1.5 about 7%, and 2.0 about 2.7%. This is the mode that works without specially trained weights.
• top-k (SLA): a fixed keep_percent of key blocks everywhere. This matches what the lightx2v SLA-style turbo LoRAs are distilled against, so it is recommended only with those trained weights.
• VSA (FastVideo): cube tiling with a coarse compression branch, matching the FastH3-VSA training setup at 10% keep rate. It uses the model's to_gate_compress layers when present.

A schedule window (start_percent / end_percent, default 0.2 to 1.0) keeps the early denoise steps dense, and two H3-specific advanced options handle conditioning: sink_conditioning attends the packed text/audio/reference rows exactly (~3% cost) so prompts and audio do not degrade, and extra_tokens adds the top-scoring tokens beyond the selected blocks to stay closer to dense.

Until now this functionality existed only as separate community node packs, and community testing had shown the LoRA alone does nothing without a matching sparse backend. The core node consolidates the three approaches behind one interface, so a single workflow can switch between Sol-Attn, SLA and VSA by changing one combo box.

The Comfy Compiler benchmark from PR #15861: MiniMax H3 720p at 158 frames on an RTX 5060 shows a stable VRAM footprint and a much higher iteration rate once the memory compiler is active.

Comfy Compiler: fewer allocations, faster steps

The Comfy Compiler (PR #15861) is a two-layer compilation process:

1. The aimdo memory compiler (in comfy-aimdo 0.5.0) computes and minimizes memory allocations up front, reducing CUDA allocation calls to a one-time setup and keeping inference memory stable and constant. According to the PR, this removes allocator thrash and the racing between dynamic-VRAM pressure and CUDA's reported numbers, and it guarantees the physical VRAM peak matches the logical allocation peak instead of letting freed allocations linger in cache.
2. CUDA graphs layer on top. The stable virtual addresses produced by the memory compiler are what make the graph record safe; synchronization is only needed on the first transformer block.

The practical effect shows in the PR benchmarks: a MiniMax H3 720p 158-frame run on an RTX 5060 that previously stalled with no iteration progress completes at roughly 43 seconds per step after the change, and MiniMax Music 3 AR sampling gains about 35% step throughput on the same card.

All models that use the prefetcher plus MiniMax H3 are converted in the initial merge. WAN is listed as future work, which would raise its maximum feasible generation sizes the same way.

Before: the H3 run prepares 19.9 GB of staged weights and then goes silent for several minutes. After the change, the same run shows steady per-step progress.

The catch: early OOM and cancel crashes

Community testing in the Banodoco Discord flagged two rough edges in the first days. Sparse runs can OOM when the Comfy Compiler is also active, and cancelling a run while the sparse node is loaded can crash ComfyUI. A follow-up commit (#16148, "Pause comfy compiler for long lived sparse allocations") already addressed the interaction, and --disable-comfy-compiler remains available as a fallback flag. The node's dense fallback path means that if a sequence falls outside the schedule window or under min_tokens, it simply runs the normal dense backend with no rewiring.

Availability

Both features are in current ComfyUI builds: the Block Sparse Attention node requires comfy-kitchen 0.2.33 (already in requirements.txt), and the Comfy Compiler is part of core with comfy-aimdo 0.5.2. No extra installation steps are needed on up-to-date installs; users on the MiniMax H3 workflows just drop the node between the model loader and the sampler.