Code Monkey home page Code Monkey logo

comfyui-dragnuwa's Introduction

This is an implementation of DragNUWA for ComfyUI

DragNUWA: DragNUWA enables users to manipulate backgrounds or objects within images directly, and the model seamlessly translates these actions into camera movements or object motions, generating the corresponding video.

Install

  1. Clone this repo into custom_nodes directory of ComfyUI location

  2. Run pip install -r requirements.txt

  3. Download the weights of DragNUWA drag_nuwa_svd.pth and put it to ComfyUI/models/checkpoints/drag_nuwa_svd.pth

For chinese users:drag_nuwa_svd.pth

smaller and faster fp16 model: dragnuwa-svd-pruned.fp16.safetensors from https://github.com/painebenjamin/app.enfugue.ai

For chinese users: wget https://hf-mirror.com/benjamin-paine/dragnuwa-pruned-safetensors/resolve/main/dragnuwa-svd-pruned.fp16.safetensors 不能直接在浏览器下载,或者参照 https://hf-mirror.com/ 官方使用说明

Tools

Motion Traj Tool Generate motion trajectories

Examples

  1. basic workflow

https://github.com/chaojie/ComfyUI-DragNUWA/blob/main/workflow.json

  1. InstantCameraMotionBrush & InstantObjectMotionBrush

InstantCameraMotionBrush Node Generate zoomin/zoomout/left/right/up/bottom camera motion brush

InstantObjectMotionBrush Node Generate zoomin/zoomout/left/right/up/bottom object motion brush (by draw mask on object)

https://github.com/chaojie/ComfyUI-DragNUWA/blob/main/workflow_InstantMotionBrush.json

  1. optical flow workflow

Thanks for Fannovol16's Unimatch_ OptFlowPreprocessor Thanks for toyxyz's load optical flow from directory

https://github.com/chaojie/ComfyUI-DragNUWA/blob/main/workflow_optical_flow.json

  1. motion brush

need nodes "https://github.com/chaojie/ComfyUI-RAFT"

https://github.com/chaojie/ComfyUI-DragNUWA/blob/main/workflow_motionbrush.json

comfyui-dragnuwa's People

Contributors

chaojie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

comfyui-dragnuwa's Issues

依赖在不同环境下有比较大的问题

比如 tokenizers==0.12.1,在 rust 编译器 1.7x 环境下会出现以下错误:

casting `&T` to `&mut T` is undefined behavior, even if the reference is unused, consider instead using an `UnsafeCell`

原因在这里: huggingface/tokenizers#1402

比如 torch>=2.0.1 ,但是在一些依赖中强制指定了 torch!=2.0.1,>=1.13.1
同时 torchaudio也会出现 libtorchaudio.so: undefined symbol: _ZN3c104cuda9SetDeviceEi 错误。

以上都会造成这个问题: #7

这是我在我的conda 环境中修正后运行正确的的依赖版本:

black==23.7.0
chardet==5.1.0
clip @ git+https://github.com/openai/CLIP.git
einops>=0.6.1
fairscale>=0.4.13
fire>=0.5.0
fsspec>=2023.6.0
invisible-watermark>=0.2.0
kornia==0.6.9
matplotlib>=3.7.2
natsort>=8.4.0
ninja>=1.11.1
numpy>=1.24.4
omegaconf>=2.3.0
open-clip-torch>=2.20.0
opencv-python==4.6.0.66
pandas>=2.0.3
pillow>=9.5.0
pudb>=2022.1.3
pytorch-lightning==2.0.1
pyyaml>=6.0.1
scipy>=1.10.1
streamlit>=0.73.1
tensorboardx==2.6
timm>=0.9.2
tokenizers==0.15.0
torch>=2.0.2
torchaudio>=2.0.2
torchdata>=0.6.1
torchmetrics>=1.0.1
torchvision>=0.15.2
tqdm>=4.65.0
transformers>=4.19.1
urllib3<1.27,>=1.25.4
wandb>=0.15.6
webdataset>=0.2.33
wheel>=0.41.0
xformers>=0.0.20
colorlog 
deepdish  
json_lines 
msgpack 
configobj 
psutil 
h5py 
imageio 
tensorboard 
moviepy 
pyyaml 
pandas 
einops

關於it may disrupt your installation environment.警告

安裝時管理器有提示
Due to changes in the torch package and versions of many other packages,
it may disrupt your installation environment.

我的理解是這個環境跟其他節點要求的不相容
能改成比較不會出問題的做法嗎?
不要影響其他節點所需環境

Followed all the steps and installed all the processes, but still reported this error

Traceback (most recent call last):
File "/data03/lxl_workspace/ComfyUI/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/data03/lxl_workspace/ComfyUI/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/data03/lxl_workspace/ComfyUI/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/data03/lxl_workspace/ComfyUI/custom_nodes/ComfyUI-DragNUWA/nodes.py", line 223, in load_dragnuwa
DragNUWA_net = Drag("cuda:0", ckpt_path, f'{comfy_path}/custom_nodes/ComfyUI-DragNUWA/DragNUWA_net.py', height, width, model_length)
File "/data03/lxl_workspace/ComfyUI/custom_nodes/ComfyUI-DragNUWA/nodes.py", line 74, in init
drag_nuwa_net = Net(args)
File "/data03/lxl_workspace/ComfyUI/custom_nodes/ComfyUI-DragNUWA/DragNUWA_net.py", line 250, in init
self.conditioner = GeneralConditioner(args.conditioner_emb_models)
File "/data03/lxl_workspace/ComfyUI/custom_nodes/ComfyUI-DragNUWA/dragnuwa/svd/modules/encoders/modules.py", line 79, in init
embedder = instantiate_from_config(embconfig)
File "/data03/lxl_workspace/ComfyUI/custom_nodes/ComfyUI-DragNUWA/dragnuwa/svd/util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "/data03/lxl_workspace/ComfyUI/custom_nodes/ComfyUI-DragNUWA/dragnuwa/svd/modules/encoders/modules.py", line 1040, in init
self.open_clip = instantiate_from_config(open_clip_embedding_config)
File "/data03/lxl_workspace/ComfyUI/custom_nodes/ComfyUI-DragNUWA/dragnuwa/svd/util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "/data03/lxl_workspace/ComfyUI/custom_nodes/ComfyUI-DragNUWA/dragnuwa/svd/modules/encoders/modules.py", line 592, in init
model, _, _ = open_clip.create_model_and_transforms(
File "/data03/lxl_workspace/miniconda3/envs/comfy-new/lib/python3.10/site-packages/open_clip/factory.py", line 382, in create_model_and_transforms
model = create_model(
File "/data03/lxl_workspace/miniconda3/envs/comfy-new/lib/python3.10/site-packages/open_clip/factory.py", line 281, in create_model
checkpoint_path = download_pretrained(pretrained_cfg, cache_dir=cache_dir)
File "/data03/lxl_workspace/miniconda3/envs/comfy-new/lib/python3.10/site-packages/open_clip/pretrained.py", line 552, in download_pretrained
target = download_pretrained_from_hf(model_id, cache_dir=cache_dir)
File "/data03/lxl_workspace/miniconda3/envs/comfy-new/lib/python3.10/site-packages/open_clip/pretrained.py", line 522, in download_pretrained_from_hf
cached_file = hf_hub_download(model_id, filename, revision=revision, cache_dir=cache_dir)
File "/data03/lxl_workspace/miniconda3/envs/comfy-new/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/data03/lxl_workspace/miniconda3/envs/comfy-new/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1377, in hf_hub_download
raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

DragNUWA weight link on GoogleDrive is dead

Looks like too many people downloaded the weights as we cannot download them anymore. Here is the message we get:


Sorry, you can't view or download this file at this time.

Too many users have viewed or downloaded this file recently. Please try accessing the file again later. If the file you are trying to access is particularly large or is shared with many people, it may take up to 24 hours to be able to view or download the file. If you still can't access a file after 24 hours, contact your domain administrator.

环境依赖问题:DLL load failed while importing flash_attn_2_cuda

如题,其他环境都安装上了,flash_attn组件也尝试重新安装,但是这个flash_attn是没解决。

python 3.10
cude:118
GPU:3080Ti

Logs
Traceback (most recent call last):
File "E:\AIGC\ComfyUI_windows\ComfyUI\nodes.py", line 1800, in load_custom_node
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA_init
.py", line 1, in
from .nodes import NODE_CLASS_MAPPINGS
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 12, in
from .DragNUWA_net import Net, args
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\DragNUWA_net.py", line 5, in
from .dragnuwa.svd.modules.diffusionmodules.video_model_flow import VideoUNet_flow, VideoResBlock_Embed
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules_init_.py", line 1, in
from .encoders.modules import GeneralConditioner
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 6, in
import kornia
File "E:\Python\Python310\lib\site-packages\kornia_init_.py", line 8, in
from . import augmentation, color, contrib, core, enhance, feature, io, losses, metrics, morphology, tracking, utils, x
File "E:\Python\Python310\lib\site-packages\kornia\augmentation_init_.py", line 2, in
from kornia.augmentation import auto, container
File "E:\Python\Python310\lib\site-packages\kornia\augmentation\auto_init_.py", line 1, in
from .autoaugment import AutoAugment
File "E:\Python\Python310\lib\site-packages\kornia\augmentation\auto\autoaugment_init_.py", line 1, in
from .autoaugment import AutoAugment
File "E:\Python\Python310\lib\site-packages\kornia\augmentation\auto\autoaugment\autoaugment.py", line 5, in
from kornia.augmentation.auto.base import SUBPLOLICY_CONFIG, PolicyAugmentBase
File "E:\Python\Python310\lib\site-packages\kornia\augmentation\auto\base.py", line 5, in
from kornia.augmentation.auto.operations.base import OperationBase
File "E:\Python\Python310\lib\site-packages\kornia\augmentation\auto\operations_init_.py", line 3, in
from .policy import PolicySequential
File "E:\Python\Python310\lib\site-packages\kornia\augmentation\auto\operations\policy.py", line 7, in
from kornia.augmentation.container.base import ImageSequentialBase, TransformMatrixMinIn
File "E:\Python\Python310\lib\site-packages\kornia\augmentation\container_init_.py", line 1, in
from kornia.augmentation.container.augment import AugmentationSequential
File "E:\Python\Python310\lib\site-packages\kornia\augmentation\container\augment.py", line 4, in
from kornia.augmentation.2d.base import RigidAffineAugmentationBase2D
File "E:\Python\Python310\lib\site-packages\kornia\augmentation_2d_init
.py", line 2, in
from kornia.augmentation.2d.intensity import *
File "E:\Python\Python310\lib\site-packages\kornia\augmentation_2d\intensity_init
.py", line 21, in
from kornia.augmentation.2d.intensity.plasma import RandomPlasmaBrightness, RandomPlasmaContrast, RandomPlasmaShadow
File "E:\Python\Python310\lib\site-packages\kornia\augmentation_2d\intensity\plasma.py", line 5, in
from kornia.contrib import diamond_square
File "E:\Python\Python310\lib\site-packages\kornia\contrib_init
.py", line 15, in
from .image_stitching import ImageStitcher
File "E:\Python\Python310\lib\site-packages\kornia\contrib\image_stitching.py", line 7, in
from kornia.feature import LocalFeatureMatcher, LoFTR
File "E:\Python\Python310\lib\site-packages\kornia\feature_init_.py", line 6, in
from .integrated import (
File "E:\Python\Python310\lib\site-packages\kornia\feature\integrated.py", line 16, in
from .lightglue import LightGlue
File "E:\Python\Python310\lib\site-packages\kornia\feature\lightglue.py", line 16, in
from flash_attn.modules.mha import FlashCrossAttention
File "E:\Python\Python310\lib\site-packages\flash_attn_init_.py", line 3, in
from flash_attn.flash_attn_interface import (
File "E:\Python\Python310\lib\site-packages\flash_attn\flash_attn_interface.py", line 10, in
import flash_attn_2_cuda as flash_attn_cuda
ImportError: DLL load failed while importing flash_attn_2_cuda: 找不到指定的模块。

Cannot import E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA module for custom nodes: DLL load failed while importing flash_attn_2_cuda: 找不到指定的模块。

Error occurred when executing Load CheckPoint DragNUWA:

Error occurred when executing Load CheckPoint DragNUWA:

An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

No module named 'custom_nodes.ComfyUI-DragNUWA'

Error occurred when executing Load CheckPoint DragNUWA:

No module named 'custom_nodes.ComfyUI-DragNUWA'

File "/mnt/data/wangxi/Comfyui/execution.py", line 154, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/mnt/data/wangxi/Comfyui/execution.py", line 84, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/mnt/data/wangxi/Comfyui/execution.py", line 77, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/mnt/data/wangxi/Comfyui/custom_nodes/ComfyUI-DragNUWA-main/nodes.py", line 338, in load_dragnuwa
DragNUWA_net = Drag("cuda:0", ckpt_path, f'{comfy_path}/custom_nodes/ComfyUI-DragNUWA/DragNUWA_net.py', height, width, model_length)
File "/mnt/data/wangxi/Comfyui/custom_nodes/ComfyUI-DragNUWA-main/nodes.py", line 111, in init
drag_nuwa_net = Net(args)
File "/mnt/data/wangxi/Comfyui/custom_nodes/ComfyUI-DragNUWA-main/DragNUWA_net.py", line 246, in init
self.denoiser = Denoiser(**args.denoiser_config)
File "/mnt/data/wangxi/Comfyui/custom_nodes/ComfyUI-DragNUWA-main/dragnuwa/svd/modules/diffusionmodules/denoiser.py", line 15, in init
self.scaling: DenoiserScaling = instantiate_from_config(scaling_config)
File "/mnt/data/wangxi/Comfyui/custom_nodes/ComfyUI-DragNUWA-main/dragnuwa/svd/util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "/mnt/data/wangxi/Comfyui/custom_nodes/ComfyUI-DragNUWA-main/dragnuwa/svd/util.py", line 186, in get_obj_from_str
return getattr(importlib.import_module(module, package=None), cls)
File "/home/wangxi/miniconda3/envs/comfyui/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1004, in _find_and_load_unlocked

image

Error running DragRun node

I downloaded the model and placed it in the right location, but I still get this error:

'''' 0%| | 0/1 [00:00<?, ?it/s]
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 251, in run_inference
return model.run(image_pil, tracking_points, inference_batch_size, motion_bucket_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 185, in run
outputs = self.forward_sample(
^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 108, in forward_sample
c, uc = self.drag_nuwa_net.conditioner.get_unconditional_conditioning(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 180, in get_unconditional_conditioning
c = self(batch_c, force_cond_zero_embeddings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 133, in forward
emb_out = embedder(batch[embedder.input_key])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 1013, in forward
out = self.encoder.encode(vid[n * n_samples : (n + 1) * n_samples])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\models\autoencoder.py", line 472, in encode
z = self.encoder(x)
^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\model.py", line 594, in forward
h = self.mid.attn_1(h)
^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in wrapped_call_impl
return self.call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\model.py", line 263, in forward
h
= self.attention(h
)
^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\model.py", line 249, in attention
out = xformers.ops.memory_efficient_attention(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha_init
.py", line 223, in memory_efficient_attention
return memory_efficient_attention(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha_init
.py", line 321, in _memory_efficient_attention
return memory_efficient_attention_forward(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha_init
.py", line 337, in _memory_efficient_attention_forward
op = _dispatch_fw(inp, False)
^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 120, in _dispatch_fw
return _run_priority_list(
^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 63, in _run_priority_list
raise NotImplementedError(msg)
NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 2880, 1, 512) (torch.float32)
key : shape=(1, 2880, 1, 512) (torch.float32)
value : shape=(1, 2880, 1, 512) (torch.float32)
attn_bias : <class 'NoneType'>
p : 0.0
decoderF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 128
xFormers wasn't build with CUDA support
attn_bias type is <class 'NoneType'>
operator wasn't built - see python -m xformers.info for more info
[email protected] is not supported because:
max(query.shape[-1] != value.shape[-1]) > 256
xFormers wasn't build with CUDA support
dtype=torch.float32 (supported: {torch.float16, torch.bfloat16})
operator wasn't built - see python -m xformers.info for more info
tritonflashattF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 128
xFormers wasn't build with CUDA support
dtype=torch.float32 (supported: {torch.float16, torch.bfloat16})
operator wasn't built - see python -m xformers.info for more info
triton is not available
cutlassF is not supported because:
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
smallkF is not supported because:
max(query.shape[-1] != value.shape[-1]) > 32
xFormers wasn't build with CUDA support
operator wasn't built - see python -m xformers.info for more info
unsupported embed per head: 512

Prompt execu'''

open_clip_pytorch_model.bin这个文件应该放在哪里

requests.exceptions.ConnectTimeout: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /laion/CLIP-ViT-H-14-laion2B-s32B-b79K/resolve/main/open_clip_pytorch_model.bin (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001C275887010>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 0e837b02-1fee-4a3b-9f00-f3b192c43501)')

我运行报错 说是连不上huggingface 虽然我是全局代理 然后我去huggingface把这个文件下载下来不知道放在哪

node import failed, any thoughts?

I still get (IMPORT FAILED): D:__AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA when I run comfyUi, any thoughts?

image

workflow_InstantMotionBrush.json报错: Specify the reduction dim with the 'dim' argument

执行第二个工作流 workflow_InstantMotionBrush.json 的时候报错

ERROR:root:Traceback (most recent call last):
File "D:\Python310\stable\ComfyUI\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Python310\stable\ComfyUI\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Python310\stable\ComfyUI\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Python310\stable\ComfyUI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 600, in run_inference
boxes = masks_to_boxes(brush_mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Python310\stable\ComfyUI\python_embeded\Lib\site-packages\torchvision\ops\boxes.py", line 412, in masks_to_boxes
bounding_boxes[index, 0] = torch.min(x)
^^^^^^^^^^^^
RuntimeError: min(): Expected reduction dim to be specified for input.numel() == 0. Specify the reduction dim with the 'dim' argument.

Comfy UI segmentation fault

Comfy UI closes with segmentation fault(core dumped) after importing this. Tried in a fresh comfy ui install with only ComfyUI-DragNUWA extension

Any idea about how to fix this

The node is in the menu and can be displayed, but run and

Error occurred when executing Load CheckPoint DragNUWA:

No module named 'custom_nodes.ComfyUI-DragNUWA'

File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 154, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 84, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 77, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 223, in load_dragnuwa
DragNUWA_net = Drag("cuda:0", ckpt_path, f'{comfy_path}/custom_nodes/ComfyUI-DragNUWA/DragNUWA_net.py', height, width, model_length)
File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 74, in init
drag_nuwa_net = Net(args)
File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\DragNUWA_net.py", line 246, in init
self.denoiser = Denoiser(**args.denoiser_config)
File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\denoiser.py", line 15, in init
self.scaling: DenoiserScaling = instantiate_from_config(scaling_config)
File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "C:\Downloads\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\util.py", line 186, in get_obj_from_str
return getattr(importlib.import_module(module, package=None), cls)
File "importlib_init_.py", line 126, in import_module
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1004, in _find_and_load_unlocked

different aspect ratios + resolutions?

Thank you so so so much for this!!!

Re the trajectory tool- very excited for auto trajectory 🙌🙌🙌 ~ in the meantime, is there any way to work with different resolutions / aspect ratios becides wide screen? how could I do a 1024x1024 video? Is there any way to import an image as a trajectory so that we can make trajectories similarly to how we would create masks in photoshop?

MPS Support

Any plans for an MPS support? Would love to use it on my Apple Silicon Mac. Thank you!

Error occurred when executing DragNUWA Run:

I get an error message when running workflow.
What can I do to solve it?

Error occurred when executing DragNUWA Run:

name 'xformers' is not defined

File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 155, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 85, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 78, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 528, in run_inference
return model.run_2(image_pil, tracking_points, inference_batch_size, motion_bucket_id, use_optical_flow, directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 340, in run_2
outputs = self.forward_sample(
^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 146, in forward_sample
c, uc = self.drag_nuwa_net.conditioner.get_unconditional_conditioning(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 180, in get_unconditional_conditioning
c = self(batch_c, force_cond_zero_embeddings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 133, in forward
emb_out = embedder(batch[embedder.input_key])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 1013, in forward
out = self.encoder.encode(vid[n * n_samples : (n + 1) * n_samples])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\models\autoencoder.py", line 472, in encode
z = self.encoder(x)
^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\model.py", line 594, in forward
h = self.mid.attn_1(h)
^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self.call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\model.py", line 263, in forward
h
= self.attention(h
)
^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\model.py", line 249, in attention
out = xformers.ops.memory_efficient_attention(
^^^^^^^^


PixPin_2024-01-24_23-31-19

If I install this, is there a way to revert the pytorch env to its original form?

The node is flag in the manager as (and I quote) "Due to changes in the torch package and versions of many other packages, it may disrupt your installation environment. "
Let's imagine i install this in prod and it accidently switches pytorch to a version i can no longer use with other nodes, will i be able to revert pytorch (and other changes) back to original by uninstalling through the manager or are we looking at something more complex?
Thank you.

Max retries exceeded with url

出现下面的错误代码,看起来是需要调用 CLIP-ViT-H-14-laion2B-s32B-b79K 这个 clip_vision的模型,但是这个我已经下载到本地了,是否可以存放在本地进行调用?

(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /laion/CLIP-ViT-H-14-laion2B-s32B-b79K/resolve/main/open_clip_pytorch_model.bin (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, '远程主机强迫关闭了一个现有的连接。', None, 10054, None)))"), '(Request ID: 97b952e0-3003-4d7c-8f5f-47417dd5fe9f)')

Traceback (most recent call last):
File "E:\AIGC\ComfyUI_windows\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "E:\AIGC\ComfyUI_windows\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "E:\AIGC\ComfyUI_windows\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 223, in load_dragnuwa
DragNUWA_net = Drag("cuda:0", ckpt_path, f'{comfy_path}/custom_nodes/ComfyUI-DragNUWA/DragNUWA_net.py', height, width, model_length)
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 74, in init
drag_nuwa_net = Net(args)
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\DragNUWA_net.py", line 250, in init
self.conditioner = GeneralConditioner(args.conditioner_emb_models)
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 79, in init
embedder = instantiate_from_config(embconfig)
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 1040, in init
self.open_clip = instantiate_from_config(open_clip_embedding_config)
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "E:\AIGC\ComfyUI_windows\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\encoders\modules.py", line 592, in init
model, _, _ = open_clip.create_model_and_transforms(
File "E:\Python\Python310\lib\site-packages\open_clip\factory.py", line 382, in create_model_and_transforms
model = create_model(
File "E:\Python\Python310\lib\site-packages\open_clip\factory.py", line 281, in create_model
checkpoint_path = download_pretrained(pretrained_cfg, cache_dir=cache_dir)
File "E:\Python\Python310\lib\site-packages\open_clip\pretrained.py", line 552, in download_pretrained
target = download_pretrained_from_hf(model_id, cache_dir=cache_dir)
File "E:\Python\Python310\lib\site-packages\open_clip\pretrained.py", line 522, in download_pretrained_from_hf
cached_file = hf_hub_download(model_id, filename, revision=revision, cache_dir=cache_dir)
File "E:\Python\Python310\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "E:\Python\Python310\lib\site-packages\huggingface_hub\file_download.py", line 1238, in hf_hub_download
metadata = get_hf_file_metadata(
File "E:\Python\Python310\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "E:\Python\Python310\lib\site-packages\huggingface_hub\file_download.py", line 1631, in get_hf_file_metadata
r = _request_wrapper(
File "E:\Python\Python310\lib\site-packages\huggingface_hub\file_download.py", line 385, in _request_wrapper
response = _request_wrapper(
File "E:\Python\Python310\lib\site-packages\huggingface_hub\file_download.py", line 408, in _request_wrapper
response = get_session().request(method=method, url=url, **params)
File "E:\Python\Python310\lib\site-packages\requests\sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "E:\Python\Python310\lib\site-packages\requests\sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "E:\Python\Python310\lib\site-packages\huggingface_hub\utils_http.py", line 67, in send
return super().send(request, *args, **kwargs)
File "E:\Python\Python310\lib\site-packages\requests\adapters.py", line 513, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /laion/CLIP-ViT-H-14-laion2B-s32B-b79K/resolve/main/open_clip_pytorch_model.bin (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, '远程主机强迫关闭了一个现有的连接。', None, 10054, None)))"), '(Request ID: 3d361a64-b48c-4a8e-b386-030ce40d5506)')a

AssertionError when running optical flow workflow

ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "E:\IMAGE\ComfyUI_test\ComfyUI\execution.py", line 155, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\execution.py", line 85, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\execution.py", line 78, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 498, in run_inference
return model.run_brush(image_pil, motion_brush, inference_batch_size, motion_bucket_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 380, in run_brush
outputs = self.forward_sample(
^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 175, in forward_sample
samples_z = self.drag_nuwa_net.sampler(denoiser, randn, cond=c, uc=uc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\sampling.py", line 120, in call
x = self.sampler_step(
^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\sampling.py", line 99, in sampler_step
denoised = self.denoise(x, denoiser, sigma_hat, cond, uc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\sampling.py", line 55, in denoise
denoised = denoiser(*self.guider.prepare_inputs(x, sigma, cond, uc))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 173, in denoiser
return self.drag_nuwa_net.denoiser(self.drag_nuwa_net.model, input, sigma, c, **additional_model_inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\denoiser.py", line 37, in forward
network(input * c_in, c_noise, cond, **additional_model_inputs) * c_out
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\wrappers.py", line 28, in forward
return self.diffusion_model(
^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\IMAGE\ComfyUI_test\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\video_model_flow.py", line 1270, in forward
assert y.shape[0] == x.shape[0]
^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
image

Bad default workflow

Error occurred when executing DragNUWA Run:



File "E:\AI\ComfyUI\execution.py", line 155, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "E:\AI\ComfyUI\execution.py", line 85, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "E:\AI\ComfyUI\execution.py", line 78, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 251, in run_inference
return model.run(image_pil, tracking_points, inference_batch_size, motion_bucket_id)
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 184, in run
outputs = self.forward_sample(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 136, in forward_sample
samples_z = self.drag_nuwa_net.sampler(denoiser, randn, cond=c, uc=uc)
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\sampling.py", line 120, in __call__
x = self.sampler_step(
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\sampling.py", line 99, in sampler_step
denoised = self.denoise(x, denoiser, sigma_hat, cond, uc)
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\sampling.py", line 55, in denoise
denoised = denoiser(*self.guider.prepare_inputs(x, sigma, cond, uc))
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\nodes.py", line 134, in denoiser
return self.drag_nuwa_net.denoiser(self.drag_nuwa_net.model, input, sigma, c, **additional_model_inputs)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\denoiser.py", line 37, in forward
network(input * c_in, c_noise, cond, **additional_model_inputs) * c_out
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\wrappers.py", line 28, in forward
return self.diffusion_model(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "E:\AI\ComfyUI\custom_nodes\ComfyUI-DragNUWA\dragnuwa\svd\modules\diffusionmodules\video_model_flow.py", line 1270, in forward
assert y.shape[0] == x.shape[0]

No module named 'custom_nodes.ComfyUI-DragNUWA'

Error occurred when executing Load CheckPoint DragNUWA:

No module named 'custom_nodes.ComfyUI-DragNUWA'

File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 155, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 85, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 78, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA-main\nodes.py", line 338, in load_dragnuwa
DragNUWA_net = Drag("cuda:0", ckpt_path, f'{comfy_path}/custom_nodes/ComfyUI-DragNUWA/DragNUWA_net.py', height, width, model_length)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA-main\nodes.py", line 111, in init
drag_nuwa_net = Net(args)
^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA-main\DragNUWA_net.py", line 246, in init
self.denoiser = Denoiser(**args.denoiser_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA-main\dragnuwa\svd\modules\diffusionmodules\denoiser.py", line 15, in init
self.scaling: DenoiserScaling = instantiate_from_config(scaling_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA-main\dragnuwa\svd\util.py", line 175, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-DragNUWA-main\dragnuwa\svd\util.py", line 186, in get_obj_from_str
return getattr(importlib.import_module(module, package=None), cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "importlib_init_.py", line 126, in import_module
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1126, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1140, in _find_and_load_unlocked

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.