Load LoRA
Documentation
- Class name:
LoraLoader
- Category:
loaders
- Output node:
False
The LoraLoader node is designed to dynamically load and apply LoRA (Low-Rank Adaptation) adjustments to models and CLIP instances based on specified strengths and LoRA file names. It facilitates the customization of pre-trained models by applying fine-tuned adjustments without altering the original model weights directly, enabling more flexible and targeted model behavior modifications.
Input types
Required
model
- The model to which LoRA adjustments will be applied. It's crucial for customizing the model's behavior without changing its original structure. The choice of model directly influences the effectiveness and applicability of the LoRA adjustments, as different models may respond differently to the same set of adjustments.
- Comfy dtype:
MODEL
- Python dtype:
torch.nn.Module
clip
- The CLIP instance to which LoRA adjustments will be applied, allowing for customized behavior in processing visual and textual data. The adjustments can significantly alter how the CLIP model processes and interprets visual and textual inputs, thereby affecting the outcomes of tasks like image captioning or text-to-image generation.
- Comfy dtype:
CLIP
- Python dtype:
torch.nn.Module
lora_name
- The name of the LoRA file containing the adjustments to be applied. This enables the selection of specific fine-tuning adjustments for the model and CLIP instance. The specific LoRA file chosen dictates the nature of the adjustments and can lead to varied enhancements or modifications in model performance.
- Comfy dtype:
COMBO[STRING]
- Python dtype:
str
strength_model
- Determines the intensity of the LoRA adjustments applied to the model. This allows for fine-grained control over the extent of model customization. Higher strengths mean more pronounced adjustments, which can lead to significant changes in model behavior, potentially improving performance on specific tasks.
- Comfy dtype:
FLOAT
- Python dtype:
float
strength_clip
- Determines the intensity of the LoRA adjustments applied to the CLIP instance. This allows for fine-grained control over the extent of CLIP customization. Similar to the model, higher strengths result in more noticeable changes, affecting how the CLIP model processes data.
- Comfy dtype:
FLOAT
- Python dtype:
float
Output types
model
- Comfy dtype:
MODEL
- The model with LoRA adjustments applied, reflecting the specified customization. The adjustments can enhance the model's performance on specific tasks or alter its behavior to better suit particular applications.
- Python dtype:
torch.nn.Module
- Comfy dtype:
clip
- Comfy dtype:
CLIP
- The CLIP instance with LoRA adjustments applied, reflecting the specified customization. These adjustments can lead to improved or altered performance in tasks involving visual and textual data processing.
- Python dtype:
torch.nn.Module
- Comfy dtype:
Usage tips
- Infra type:
GPU
- Common nodes:
- LoraLoader
- CLIPTextEncode
- Reroute
- VideoLinearCFGGuidance
- KSampler
- FaceDetailer
- ModelSamplingDiscrete
- ADE_AnimateDiffLoaderWithContext
- KSampler //Inspire
- ToBasicPipe
Source code
class LoraLoader:
def __init__(self):
self.loaded_lora = None
@classmethod
def INPUT_TYPES(s):
return {"required": { "model": ("MODEL",),
"clip": ("CLIP", ),
"lora_name": (folder_paths.get_filename_list("loras"), ),
"strength_model": ("FLOAT", {"default": 1.0, "min": -20.0, "max": 20.0, "step": 0.01}),
"strength_clip": ("FLOAT", {"default": 1.0, "min": -20.0, "max": 20.0, "step": 0.01}),
}}
RETURN_TYPES = ("MODEL", "CLIP")
FUNCTION = "load_lora"
CATEGORY = "loaders"
def load_lora(self, model, clip, lora_name, strength_model, strength_clip):
if strength_model == 0 and strength_clip == 0:
return (model, clip)
lora_path = folder_paths.get_full_path("loras", lora_name)
lora = None
if self.loaded_lora is not None:
if self.loaded_lora[0] == lora_path:
lora = self.loaded_lora[1]
else:
temp = self.loaded_lora
self.loaded_lora = None
del temp
if lora is None:
lora = comfy.utils.load_torch_file(lora_path, safe_load=True)
self.loaded_lora = (lora_path, lora)
model_lora, clip_lora = comfy.sd.load_lora_for_models(model, clip, lora, strength_model, strength_clip)
return (model_lora, clip_lora)