Start Recording CUDAMemory History(StartRecordCUDAMemoryHistory)
This node invokes PyTorch's torch。
Start Record CUDA Memory History
The Start Recording CUDAMemory History node initiates tracking and recording of CUDA memory allocation and deallocation history. This node always runs, regardless of bypass or mute status. It is a diagnostic tool designed to help users gain in-depth insights into GPU memory usage during model runtime, providing a data foundation for subsequent performance analysis and memory leak investigation. This node is typically used in conjunction with the EndRecordCUDAMemoryHistory node to complete a full recording cycle.
Node Functionality
This node implements its core functionality by calling PyTorch's torch.cuda.memory._record_memory_history interface. Users can configure the recording scope (enabled), the level of detail for call stack information (context and stacks), and the maximum number of entries to record (max_entries). Before starting the recording, the node performs a soft cache clearing and resets peak memory statistics to ensure the accuracy of the recorded data.
Node Parameter Description - Start Recording CUDAMemory History
Connection Inputs (Inputs)
| Parameter Name | Data Type | Required | Default Value | Value Range/Options | Description |
|---|---|---|---|---|---|
input | IO | Yes | - | - | Passes the received input data unchanged to downstream nodes, maintaining workflow connectivity. |
Control Parameters (Parameters)
| Parameter Name | Data Type | Required | Default Value | Value Range/Options | Description |
|---|---|---|---|---|---|
enabled | COMBO | Yes | all | "all", "state", "None" | Controls the scope of memory history recording. Optional values are "all" (record all memory events), "state" (record information only for allocated memory), or "None" (disable recording). |
context | COMBO | Yes | all | "all", "state", "alloc", "None" | Controls which memory operations have their call stacks recorded. Optional values are "all" (record for all operations), "alloc" (record for allocation calls), "state" (record for allocated memory), or "None" (do not record call stacks). |
stacks | COMBO | Yes | python | "python", "all" | Determines the level of detail for call stack information recording. Optional values are "python" (record only Python/TorchScript/inductor frameworks) or "all" (also record C++ frameworks). |
max_entries | INT | Yes | 100000 | 1000 - 10000000 | Maximum number of entries to record. |
Output
| Parameter Name | Data Type | Description |
|---|---|---|
| input | ANY | Passes the received input data unchanged to downstream nodes, maintaining workflow connectivity. |
Usage Scenarios
When building a complex image generation or model inference workflow and suspecting memory leaks or wishing to optimize memory usage efficiency, you can insert this node at the beginning of the workflow. Subsequently, use the EndRecordCUDAMemoryHistory node at the end of the process to stop recording and save a snapshot file. Finally, use analysis tools (such as torch.cuda.memory._snapshot()) to view the detailed memory event history.
Notes
Please note that this node is an experimental feature, and its underlying API may change with PyTorch version updates. Enabling detailed history recording (especially with stacks set to "all") may incur some runtime performance overhead and consume additional memory to store the recorded information.
Start Recording CUDAMemory History Node Source Code Link
The Start Recording CUDAMemory History node is from the ComfyUI-KJNodes node package.
Comments
Sign in with GitHub to join the discussion.