aigodlike / comfyui-tooncrafter Goto Github PK

View Code? Open in Web Editor NEW

294.0 4.0 9.0 11.54 MB

This project is used to enable ToonCrafter to be used in ComfyUI.

License: Apache License 2.0

Python 99.70% Shell 0.30%

comfyui tooncrafter

comfyui-tooncrafter's Introduction

Introduction

This project is used to enable ToonCrafter to be used in ComfyUI.

You can use it to achieve generative keyframe animation(RTX 4090,26s)

2D.mp4

3D.mp4

And use it in Blender for animation rendering and prediction

Additionally, it can be used completely without a network

Installation

ComfyUI Custom Node

cd ComfyUI/custom_nodes
git clone https://github.com/AIGODLIKE/ComfyUI-ToonCrafter
cd ComfyUI-ToonCrafter
# install dependencies
..\..\..\python_embeded\python.exe -m pip install -r requirements.txt

Model Prepare
- Download the weights:
  - 512 full weights High VRAM usage, fp16 reccomended
  - 512 fp16 weights
- Put it in into ComfyUI-ToonCrafter\ToonCrafter\checkpoints\tooncrafter_512_interp_v1 for example 512x512.
Enjoy it!

Showcases

Blender

You can even use it directly in Blender!(ComfyUI-BlenderAI-node)

ComfyUI-BlenderAI-node.mp4

Input starting frame	Input ending frame	Generated video

comfyui-tooncrafter's People

Contributors

Stargazers

Watchers

Forkers

kustomzone fizzledorf thbmelatoe won21kr zorrock comfynodeprs comfynodeprs winnerking2010 lovehifi

comfyui-tooncrafter's Issues

Broke my comfy install, be cautious

Digs deep into torch and disables the pytorch cuda apparently, uninstalling and reinstalling everything as it goes.

If you've struggled with keeping your gpu libs balanced like I have, you'll be glad if you keep a backup env. Reinstalling everything again is a slow process

**raise AssertionError("Torch not compiled with CUDA enabled")

AssertionError: Torch not compiled with CUDA enabled**

Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions

I am getting the following error when running ToonCrafter:

CUDA error: named symbol not found
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\apps\SD-WebUI\ComfyUI\custom_nodes\ComfyUI-ToonCrafter\__init__.py", line 200, in get_image
batch_samples = batch_ddim_sampling(model, cond, noise_shape, n_samples=1, ddim_steps=steps, ddim_eta=eta, cfg_scale=cfg_scale, hs=hs, callback=cb)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter\ToonCrafter\scripts\evaluation\funcs.py", line 65, in batch_ddim_sampling
samples, _ = ddim_sampler.sample(S=ddim_steps,
File "D:\apps\Python\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\samplers\ddim.py", line 113, in sample
samples, intermediates = self.ddim_sampling(conditioning, size,
File "D:\apps\Python\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\samplers\ddim.py", line 182, in ddim_sampling
outs = self.p_sample_ddim(img, cond, ts, index=index, use_original_steps=ddim_use_original_steps,
File "D:\apps\Python\Python310\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\samplers\ddim.py", line 221, in p_sample_ddim
e_t_cond = self.model.apply_model(x, t, c, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\ddpm3d.py", line 755, in apply_model
x_recon = self.model(x_noisy, t, **cond, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\ddpm3d.py", line 1274, in forward
out = self.diffusion_model(xc, t, context=cc, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\networks\openaimodel3d.py", line 584, in forward
h = module(h, emb, context=context, batch_size=b)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\networks\openaimodel3d.py", line 41, in forward
x = layer(x, context)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\attention.py", line 304, in forward
x = block(x, context=context, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\attention.py", line 239, in forward
return checkpoint(self._forward, input_tuple, self.parameters(), self.checkpoint)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\common.py", line 94, in checkpoint
return func(*inputs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\attention.py", line 243, in _forward
x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None, mask=mask) + x
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\attention.py", line 175, in efficient_forward
out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=None)
File "D:\apps\Python\Python310\lib\site-packages\xformers\ops\fmha\__init__.py", line 247, in memory_efficient_attention
return _memory_efficient_attention(
File "D:\apps\Python\Python310\lib\site-packages\xformers\ops\fmha\__init__.py", line 365, in _memory_efficient_attention
return _memory_efficient_attention_forward(
File "D:\apps\Python\Python310\lib\site-packages\xformers\ops\fmha\__init__.py", line 385, in _memory_efficient_attention_forward
out, *_ = op.apply(inp, needs_gradient=False)
File "D:\apps\Python\Python310\lib\site-packages\xformers\ops\fmha\cutlass.py", line 202, in apply
return cls.apply_bmhk(inp, needs_gradient=needs_gradient)
File "D:\apps\Python\Python310\lib\site-packages\xformers\ops\fmha\cutlass.py", line 266, in apply_bmhk
out, lse, rng_seed, rng_offset = cls.OPERATOR(
File "D:\apps\Python\Python310\lib\site-packages\torch\_ops.py", line 755, in __call__
return self._op(*args, **(kwargs or {}))

[Enhancement] Make sliders convertible into Input Widgets

Sliders are nice, but often there is necessity to inject external values for automated workflows
Thanks

4090 OOM while using fp16 OR producing black frames

Error on a 4090 while using fp16 safetensor, I receive OOM and/or black frames.. Any ideas?

black frames:

got prompt
'🔥 - 4 Nodes not included in prompt but is activated'
LatentVisualDiffusion: Running in v-prediction mode
AE working on z of shape (1, 4, 32, 32) = 4096 dimensions.
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
making attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels
making attention of type 'memory-efficient-cross-attn-fusion' with 512 in_channels
Loaded ViT-H-14 model config.
Loading pretrained ViT-H-14 weights (laion2b_s32b_b79k).
Loaded ViT-H-14 model config.
Loading pretrained ViT-H-14 weights (laion2b_s32b_b79k).
D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-ToonCrafter\ToonCrafter\checkpoints\tooncrafter_512_interp_v1\model_512_interp-fp16.ckpt

model checkpoint loaded.
Global seed set to 129
start: a anime blinking 2024-06-02 13:29:38
!!! Exception during processing!!! Allocation on device
Traceback (most recent call last):
File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\ComfyUI\custom_nodes\ComfyUI-ToonCrafter_init_.py", line 176, in get_image
batch_samples = batch_ddim_sampling(model, cond, noise_shape, n_samples=1, ddim_steps=steps, ddim_eta=eta, cfg_scale=cfg_scale, hs=hs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA/ComfyUI/custom_nodes/ComfyUI-ToonCrafter\ToonCrafter\scripts\evaluation\funcs.py", line 79, in batch_ddim_sampling
batch_images = model.decode_first_stage(samples, **additional_decode_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\ddpm3d.py", line 683, in decode_first_stage
return self.decode_core(z, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\ddpm3d.py", line 671, in decode_core
out = self.first_stage_model.decode(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\autoencoder.py", line 119, in decode
dec = self.decoder(z, **kwargs) # change for SVD decoder by adding **kwargs
^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\autoencoder_dualref.py", line 584, in forward
h = self.up[i_level].block[i_block](h, temb, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\autoencoder_dualref.py", line 975, in forward
x = super().forward(x, temb)
^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\autoencoder_dualref.py", line 74, in forward
h = nonlinearity(h)
^^^^^^^^^^^^^^^
File "D:\COMFYUI_BETA/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\autoencoder_dualref.py", line 25, in nonlinearity
return x * torch.sigmoid(x)
~~^~~~~~~~~~~~~~~~~~
torch.cuda.OutOfMemoryError: Allocation on device

Prompt executed in 24.62 seconds

add path to model

There is fp16 version

https://huggingface.co/Kijai/DynamiCrafter_pruned/resolve/main/tooncrafter_512_interp-fp16.safetensors

Can you add path to model to the node?

4090 using 512 fp16 weights is still out of memory.

Error occurred when executing ToonCrafterNode:

Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 22.16 GiB
Requested : 1.25 GiB
Device limit : 23.99 GiB
Free (according to CUDA): 0 bytes
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB

File "E:\soft\ComfyUI-aki-v1.2\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "E:\soft\ComfyUI-aki-v1.2\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "E:\soft\ComfyUI-aki-v1.2\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "E:\soft\ComfyUI-aki-v1.2\custom_nodes\ComfyUI-ToonCrafter_init_.py", line 161, in get_image
z, hs = self.get_latent_z_with_hidden_states(model, videos)
File "E:\soft\ComfyUI-aki-v1.2\custom_nodes\ComfyUI-ToonCrafter_init_.py", line 218, in get_latent_z_with_hidden_states
encoder_posterior, hidden_states = model.first_stage_model.encode(x, return_hidden_states=True)
File "E:\soft/ComfyUI-aki-v1.2/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\autoencoder.py", line 106, in encode
h, hidden = self.encoder(x, return_hidden_states)
File "E:\soft\ComfyUI-aki-v1.2\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "E:\soft\ComfyUI-aki-v1.2\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "E:\soft/ComfyUI-aki-v1.2/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\networks\ae_modules.py", line 454, in forward
h = self.down[i_level].block[i_block](hs[-1], temb)
File "E:\soft\ComfyUI-aki-v1.2\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "E:\soft\ComfyUI-aki-v1.2\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "E:\soft/ComfyUI-aki-v1.2/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\networks\ae_modules.py", line 200, in forward
h = nonlinearity(h)
File "E:\soft/ComfyUI-aki-v1.2/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\networks\ae_modules.py", line 15, in nonlinearity
return x * torch.sigmoid(x)

Multiple ToonCrafter repos (Issue in other repo)

Hello,
I've touched base with Kijai, the author of the dynamiccrafter repo), and I would like to have some communication set up between the repos so we can decide how to proceed. You can view the issue here. Kosinkadink, the author of the animatediff evolved repo, also has plans on developing ToonCrafter (probably in his own repo) as well so I will be letting him know about the open issue as well. Let's figure this out together!

4090 out of memory

!!! Exception during processing!!! Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 19.13 GiB
Requested : 2.50 GiB
Device limit : 23.64 GiB
Free (according to CUDA): 8.81 MiB
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB
Traceback (most recent call last):
File "/workspace/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/workspace/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/workspace/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/init.py", line 157, in get_image
batch_samples = batch_ddim_sampling(model, cond, noise_shape, n_samples=1, ddim_steps=steps, ddim_eta=eta, cfg_scale=cfg_scale, hs=hs)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter/scripts/evaluation/funcs.py", line 79, in batch_ddim_sampling
batch_images = model.decode_first_stage(samples, **additional_decode_kwargs)
File "/workspace/ComfyUI/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter/lvdm/models/ddpm3d.py", line 683, in decode_first_stage
return self.decode_core(z, **kwargs)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter/lvdm/models/ddpm3d.py", line 671, in decode_core
out = self.first_stage_model.decode(
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter/lvdm/models/autoencoder.py", line 119, in decode
dec = self.decoder(z, **kwargs) # change for SVD decoder by adding **kwargs
File "/workspace/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/workspace/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter/lvdm/models/autoencoder_dualref.py", line 510, in forward
h = self.up[i_level].block[i_block](h, temb, **kwargs)
File "/workspace/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/workspace/ComfyUI/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter/lvdm/models/autoencoder_dualref.py", line 901, in forward
x = super().forward(x, temb)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter/lvdm/models/autoencoder_dualref.py", line 78, in forward
h = nonlinearity(h)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter/lvdm/models/autoencoder_dualref.py", line 29, in nonlinearity
return x * torch.sigmoid(x)
torch.cuda.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 19.13 GiB
Requested : 2.50 GiB
Device limit : 23.64 GiB
Free (according to CUDA): 8.81 MiB
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB

Prompt executed in 11.75 seconds

where is open_clip_pytorch_model.bin ?

2024-06-04 20:58:37,987- root:212- INFO- Loaded ViT-H-14 model config.
open_clip_pytorch_model.bin: 1%| | 41.9M/3.94G [03:50<5:28:52, 198kB/s]

open_clip_pytorch_model.bin: 4%|▎ | 141M/3.94G [09:14<4:09:44, 254kB/s]
2024-06-04 21:08:04,199- root:179- ERROR- !!! Exception during processing!!! Consistency check failed: file should be of size 3944692325 but has size 140870298 (open_clip_pytorch_model.bin).
We are sorry for the inconvenience. Please retry download and pass force_download=True, resume_download=False as argument.
If the issue persists, please let us know by opening an issue on https://github.com/huggingface/huggingface_hub.

No available kernel. Aborting execution

Getting the following error:

Error occurred when executing ToonCrafterNode:

No available kernel. Aborting execution.

File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\apps\SD-WebUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\apps\SD-WebUI\ComfyUI\custom_nodes\ComfyUI-ToonCrafter\__init__.py", line 143, in get_image
text_emb = model.get_learned_conditioning([prompt])
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\models\ddpm3d.py", line 602, in get_learned_conditioning
c = self.cond_stage_model.encode(c)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\encoders\condition.py", line 253, in encode
return self(text)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\encoders\condition.py", line 230, in forward
z = self.encode_with_transformer(tokens.to(self.device))
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\encoders\condition.py", line 237, in encode_with_transformer
x = self.text_transformer_forward(x, attn_mask=self.model.attn_mask)
File "D:\apps/SD-WebUI/ComfyUI/custom_nodes/ComfyUI-ToonCrafter/ToonCrafter\lvdm\modules\encoders\condition.py", line 249, in text_transformer_forward
x = r(x, attn_mask=attn_mask)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\open_clip\transformer.py", line 241, in forward
x = q_x + self.ls_1(self.attention(q_x=self.ln_1(q_x), k_x=k_x, v_x=v_x, attn_mask=attn_mask))
File "D:\apps\Python\Python310\lib\site-packages\open_clip\transformer.py", line 227, in attention
return self.attn(
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\modules\activation.py", line 1241, in forward
attn_output, attn_output_weights = F.multi_head_attention_forward(
File "D:\apps\Python\Python310\lib\site-packages\torch\nn\functional.py", line 5476, in multi_head_attention_forward
attn_output = scaled_dot_product_attention(q, k, v, attn_mask, dropout_p, is_causal)

Running ZLUDA (not CUDA - As I am on AMD GPU)
PyTorch 2.3.0+cu121 with CUDA 1201
Python 3.10.11 (you have 3.10.11)
No XFORMERS

requirements changes torch to cpu style .can't run comfy ui

before \python_embeded\python.exe -m pip install -r requirements.txt

import torch
print(torch.version)
2.2.0+cu121
torch.cuda.is_available()
True

安装依赖之后
after install requirements.txt

print(torch.version)
2.3.0+cpu
torch.cuda.is_available()
False

and can't run comfy ui：
File "D:.....\new_ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\cuda_init_.py", line 284, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
彻底打不开comfy ui了，

Where to put the Checkpoint ?

Hi,
Where to put the Checkpoint "
tooncrafter_512_interp-fp16.safetensors" please?
I try your path:
ComfyUI-ToonCrafter\ToonCrafter\checkpoints\tooncrafter_512_interp_v1
and I've try to put it everywhere else but still have this error message:
Required input is missing: ckpt_name
Thanks,
Olivier

Could not find a version that satisfies the requirement decord==0.6.0

Hi,

While trying to install the requirements.txt I get:

ComfyUI-ToonCrafter % pip install -r requirements.txt
ERROR: Could not find a version that satisfies the requirement decord==0.6.0 (from versions: none)
ERROR: No matching distribution found for decord==0.6.0

Any available workaround?