LTX2 Attention Tuner Patch(LTX2AttentionTunerPatch)

Description

EXPERIMENTAL! This node patches a LTX2 model's forward pass to apply per-modality attention scaling factors. It also reduces peak VRAM usage compared to the unpatched forward pass. Use it to fine-tune the influence of video and audio attention streams during generation.

Inputs

• model (MODEL, required) – The LTX2 model to patch. Must be a compatible model loaded in ComfyUI.

• blocks (STRING, required, default: "") – Comma‑separated list of transformer block indices to apply the patch to. Leave empty to apply to all blocks. Example: "0,2,4" would only patch blocks 0, 2, and 4.

• video_scale (FLOAT, default: 1.0, range: 0.0 – 100.0, step 0.01) – Scaling factor for video‑to‑video attention (self‑attention within the video modality).

• audio_scale (FLOAT, default: 1.0, range: 0.0 – 100.0, step 0.01) – Scaling factor for audio‑to‑audio attention (self‑attention within the audio modality).

• audio_to_video_scale (FLOAT, default: 1.0, range: 0.0 – 100.0, step 0.01) – Scaling factor for cross‑attention from audio to video. Controls how much audio information influences the video representation. (Tooltip: "Scaling factor for video attention." – likely a shorthand for the effect on video attention from audio.)

• video_to_audio_scale (FLOAT, default: 1.0, range: 0.0 – 100.0, step 0.01) – Scaling factor for cross‑attention from video to audio. Controls how much video information influences the audio representation. (Tooltip: "Scaling factor for audio attention." – corresponding shorthand.)

• triton_kernels (BOOLEAN, default: True) – When enabled, uses Triton fused kernels for the norm+scale+shift and RoPE application operations. This can be very slightly faster, but requires a compatible environment with Triton installed. Disable if you encounter errors or prefer a pure PyTorch fallback.

Outputs

• MODEL – The patched LTX2 model with the attention scaling modifications applied.

Usage Notes

• This node is experimental. Back up your workflows and test thoroughly before relying on the results.
• Because the node modifies the forward pass, the patched model will use a custom attention implementation that reduces peak VRAM consumption – useful for larger generations or when memory is constrained.
• The blocks input lets you selectively patch only certain transformer blocks. This can help isolate effects or reduce the performance impact of the patch.
• All scale factors act as multipliers on the raw attention logits before softmax. A value of 1.0 leaves the attention unchanged; values > 1.0 amplify the contribution of that modality, values < 1.0 dampen it.
• The four scaling factors (video, audio, audio→video, video→audio) give you independent control over intra‑modality and cross‑modality attention. For example, to make the video generation more sensitive to the audio track, increase audio_to_video_scale.
• If you don't need cross‑modality adjustments, leave audio_to_video_scale and video_to_audio_scale at 1.0 and only tweak video_scale and audio_scale.
• The triton_kernels option is typically safe to leave enabled; it only slightly speeds up the patched forward pass. Disable if Triton is not installed or causes compatibility issues.