facok GitHub avatar

comfyui-krea2-controlnet

facok

ComfyUI Krea2 Control is a plugin that adds native-style nodes for Krea2 Control LoRA inference, enabling control image conditioning via VAE-encoded latents while preserving the original Krea2 diffusion path.

Stars

146

7-day growth

No data

Forks

9

Open issues

6

License

No data

Last updated

2026-07-04

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It integrates control LoRA support into ComfyUI's native Krea2 workflow without breaking existing functionality, uses live module weight shapes for quantized model compatibility, and offers flexible image preprocessing options.

Who it is for

  • ComfyUI users wanting to control Krea2 generation
  • AI artists and researchers experimenting with depth/pose/canny control
  • Developers building custom workflows with Krea2 Control LoRAs
  • Users of quantized/GGUF models needing LoRA compatibility

Use cases

  • Applying depth control from Depth Anything output to Krea2 generation
  • Adding pose, canny, or lineart conditioning to Krea2 images
  • Batch processing control images for video-frame or independent samples
  • Integrating control without leaving ComfyUI's standard node graph

Strengths

  • Preserves ComfyUI's built-in Krea2 inference and regular LoRA patches
  • Block LoRA weights applied via ModelPatcher, supporting offload and low-VRAM
  • Live module weight shape reading improves compatibility with quantized loaders
  • Lightweight image options (grayscale, normalize, invert, batch mode) without heavy preprocessors

Considerations

  • Control type is determined solely by the LoRA checkpoint, not automatically detected
  • Requires explicit control latent attachment; sampling fails if omitted
  • Does not include preprocessors; control images must be prepared externally

README quick start

ComfyUI Krea2 Control

Chinese documentation: README_zh.md

Native-style ComfyUI nodes for Krea2 Control LoRA inference. The plugin keeps ComfyUI's built-in Krea2 inference path intact, uses the expanded input projection from the LoRA checkpoint during sampling, and injects a VAE-encoded control latent.

Nodes

  • Krea2 Control LoRA Loader: loads a Krea2 Control LoRA from models/loras, applies compatible block LoRA weights to the Krea2 model, prepares the expanded input projection, and registers the sampling wrapper.
  • Krea2 Control Image Encode: encodes any control IMAGE with the supplied Krea2/Qwen VAE. It can consume outputs from comfyui_controlnet_aux preprocessors such as Depth Anything, Canny, OpenPose, lineart, and normal maps, but it does not import or call comfyui_controlnet_aux.
  • Krea2 Control Apply: converts the encoded control latent into the Krea2 latent space and attaches it to the model after the Control LoRA has been loaded.

Basic Workflow

  1. Prepare a control image with normal ComfyUI nodes, or connect a comfyui_controlnet_aux preprocessor output.
  2. Encode that control image with Krea2 Control Image Encode using the Krea2/Qwen image VAE. Keep the default match_latent_size and connect the sampler latent to this node's latent input.
  3. Load the Krea2 Control LoRA with Krea2 Control LoRA Loader.
  4. Attach the encoded latent with Krea2 Control Apply.
  5. Send the resulting model to your sampler.

Krea2 Control Apply is required after the loader. If the Control LoRA is loaded without an attached control latent, sampling fails instead of silently running a partially patched model.

Block LoRA weights are applied through ComfyUI's ModelPatcher so normal model loading, offload, and low-VRAM behavior still apply. During the Krea2 diffusion forward call, image tokens still pass through the native first projection so regular LoRA patches on the base model remain active; the Control LoRA contributes only the control-token half of the expanded projection. The temporary projection state is restored immediately afterwards, so removing the node does not leave the base Krea2 path patched.

LoRA block matching reads the live module weight shapes instead of relying only on state_dict() s

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
littledivy
Featured
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning
1,482