Model Patch Torch Settings (ModelPatchTorchSettings) - ComfyUI-KJNodes

The Model Patch Torch Settings node technically works by cloning the input model and binding ON_PRE_RUN and ON_CLEANUP callback functions to it.

Model Patch Torch Settings is an experimental node that allows users to dynamically modify PyTorch's underlying computation settings before and after model execution. Its core functionality involves adding callback functions to activate specific performance optimization options before model execution and restore the original state afterward, enabling fine-grained control over the model's computational behavior.

Node Functionality

This node technically works by cloning the input model and binding ON_PRE_RUN and ON_CLEANUP callback functions to it. Currently, it primarily supports controlling the torch.backends.cuda.matmul.allow_fp16_accumulation setting, which affects whether half-precision (fp16) accumulation is allowed in CUDA matrix multiplication operations. This feature requires PyTorch 2.7.0 or a higher version (such as the nightly build) to function; otherwise, the node will report an error.

Node Parameter Description - Model Patch Torch Settings

Connection Inputs (Inputs)

Parameter Name	Data Type	Required	Default Value	Value Range/Options	Description
`model`	MODEL	Yes	-	-	Input parameter `model`: Receives a model to be processed. The node will add callback functions to modify PyTorch settings for this model.

Control Parameters (Parameters)

Parameter Name	Data Type	Required	Default Value	Value Range/Options	Description
`enable_fp16_accumulation`	BOOLEAN	Yes	False	-	Enable torch.backends.cuda.matmul.allow_fp16_accumulation, requires pytorch 2.7.0 nightly.

Output

Parameter Name	Data Type	Description
MODEL	MODEL	Output parameter `MODEL`: Outputs a new model instance with attached callback functions for use in subsequent image generation workflows.

Usage Scenarios

This node is very useful in workflows involving complex model inference or requiring optimization of VRAM usage. For example, when using a large Stable Diffusion model for image generation and wanting to try enabling fp16 accumulation to improve computation speed or reduce VRAM consumption, you can insert this node after the model loading node and before the sampler node. You can then test performance differences by toggling the Boolean parameter.

Notes

Please note that this is a node marked as "experimental," and its stability and compatibility may not be as robust as core nodes. The most critical limitation is that it must run on a PyTorch version that supports the torch.backends.cuda.matmul.allow_fp16_accumulation attribute (i.e., version 2.7.1 or higher); otherwise, the workflow will fail to execute.

Model Patch Torch Settings Node Source Code Link

The Model Patch Torch Settings node is from the ComfyUI-KJNodes node package.

Model Patch Torch Settings(ModelPatchTorchSettings)