NABLA Attention KJ (NABLA_AttentionKJ) - ComfyUI-KJNodes

The NABLA Attention KJ node implements its functionality by overriding the model's attention computation function, specifically utilizing the NABLA sparse attention algorithm.

NABLA Attention KJ is an experimental node designed to modify the attention mechanism of video generation models. Its core function is to apply NABLA sparse attention to the model, aiming to optimize memory usage and computational efficiency during the video generation process.

Node Functionality

This node works by overriding the model's attention computation function, specifically using the NABLA sparse attention algorithm. It requires a newer version of PyTorch to support the flex_attention module and is currently primarily adapted for the Kadinsky5 video model. Its internal mechanism dynamically generates a sparse attention mask based on the provided latent shape and window parameters to limit the scope of attention computation.

Node Parameter Description - NABLA Attention KJ

Connection Inputs (Inputs)

Parameter Name	Data Type	Required	Default Value	Value Range/Options	Description
`model`	MODEL	Yes	-	-	The base model whose attention mechanism needs to be modified. This parameter is used to receive a standard generation model, and the node will override its internal attention computation method.
`latent`	LATENT	Yes	-	-	Only used to get the latent shape

Control Parameters (Parameters)

Parameter Name	Data Type	Required	Default Value	Value Range/Options	Description
`window_time`	INT	Yes	11	≥ 1	Temporal attention window size
`window_width`	INT	Yes	3	≥ 1	Spatial attention window size
`window_height`	INT	Yes	3	≥ 1	Spatial attention window size
`sparsity`	FLOAT	Yes	0.9	0.0 – 1.0 (Step: 0.01)	A parameter controlling the degree of attention sparsity, ranging from 0.0 to 1.0 with a step of 0.01. A higher value indicates sparser attention, meaning more correlations are ignored during computation.
`torch_compile`	BOOLEAN	Yes	True	-	Most likely required for reasonable memory usage

Output

Parameter Name	Data Type	Description
MODEL	MODEL	The new model with NABLA sparse attention applied. This output model can be used for subsequent image or video generation steps.

Usage Scenario

In a workflow for generating coherent video sequences, you can insert this node after the model loading node. By adjusting the size of the temporal and spatial windows, you can control how many past frames the model attends to and how large an area within each frame it focuses on during video frame generation, thereby reducing computational load while ensuring temporal coherence.

Notes

Please note that this is an experimental node, and its stability and compatibility may be limited. It explicitly requires a newer version of PyTorch and is currently only confirmed to work with the Kadinsky5 model. Using it with other models may cause errors or unexpected results.

NABLA Attention KJ Node Source Code Link

The NABLA Attention KJ node is from the ComfyUI-KJNodes node package.

NABLA Attention KJ(NABLA_AttentionKJ)