jesenzhang / comfyui_streamdiffusion Goto Github PK
View Code? Open in Web Editor NEWThis is a simple implemention StreamDiffusion for ComfyUI
License: Apache License 2.0
This is a simple implemention StreamDiffusion for ComfyUI
License: Apache License 2.0
The config attributes {'skip_prk_steps': True} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file.
Traceback (most recent call last):
File "C:\Apps_2023\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_StreamDiffusion\streamdiffusion\wrapper.py", line 581, in _load_model
from polygraphy import cuda
ModuleNotFoundError: No module named 'polygraphy'
Acceleration has failed. Falling back to normal mode.
Prompt executed in 3.36 seconds
hi - any clues why this error happens? afaik polygraphy is insalled and dependencies were also downloaded
[ERROR] An error occurred during the on_prompt_handler processing
Traceback (most recent call last):
File "/home/admin188/AI/webui/ComfyUI/server.py", line 633, in trigger_on_prompt
json_data = handler(json_data)
File "/home/admin188/AI/webui/ComfyUI/custom_nodes/ComfyUI_smZNodes/smZNodes.py", line 821, in prompt_handler
if node["class_type"] == "smZ CLIPTextEncode":
KeyError: 'class_type'
ERROR:aiohttp.server:Error handling request
Traceback (most recent call last):
File "/home/admin188/anaconda3/envs/fooocus/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
resp = await request_handler(request)
File "/home/admin188/anaconda3/envs/fooocus/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in handle
resp = await handler(request)
File "/home/admin188/anaconda3/envs/fooocus/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
return await handler(request)
File "/home/admin188/AI/webui/ComfyUI/server.py", line 46, in cache_control
response: web.Response = await handler(request)
File "/home/admin188/AI/webui/ComfyUI/server.py", line 472, in post_prompt
valid = execution.validate_prompt(prompt)
File "/home/admin188/AI/webui/ComfyUI/execution.py", line 600, in validate_prompt
class = nodes.NODE_CLASS_MAPPINGS[prompt[x]['class_type']]
KeyError: 'class_type'
I think the loader is not loading models correctly from /models/vae and checkpoints.
ERROR:root:Traceback (most recent call last):
File "/home/admin188/AI/webui/ComfyUI/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/admin188/AI/webui/ComfyUI/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/admin188/AI/webui/ComfyUI/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/home/admin188/anaconda3/envs/fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/admin188/AI/webui/ComfyUI/custom_nodes/ComfyUI_StreamDiffusion/nodes.py", line 232, in sample
output = model.sample(image).permute(0, 2, 3, 1)
File "/home/admin188/AI/webui/ComfyUI/custom_nodes/ComfyUI_StreamDiffusion/streamdiffusion/wrapper.py", line 330, in sample
self.stream.sample(image)
File "/home/admin188/anaconda3/envs/fooocus/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/admin188/AI/webui/ComfyUI/custom_nodes/ComfyUI_StreamDiffusion/streamdiffusion/pipeline.py", line 540, in sample
x_0_pred_out = self.predict_x0_batch(x_t_latent)
File "/home/admin188/AI/webui/ComfyUI/custom_nodes/ComfyUI_StreamDiffusion/streamdiffusion/pipeline.py", line 443, in predict_x0_batch
x_0_pred_batch, model_pred = self.unet_step(x_t_latent, t_list)
File "/home/admin188/AI/webui/ComfyUI/custom_nodes/ComfyUI_StreamDiffusion/streamdiffusion/pipeline.py", line 356, in unet_step
model_pred = self.unet(
File "/home/admin188/anaconda3/envs/fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/admin188/anaconda3/envs/fooocus/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/admin188/anaconda3/envs/fooocus/lib/python3.10/site-packages/diffusers/models/unet_2d_condition.py", line 960, in forward
timesteps = timesteps.expand(sample.shape[0])
RuntimeError: The expanded size of the tensor (202) must match the existing size (4) at non-singleton dimension 0. Target sizes: [202]. Tensor sizes: [4]
Ok great work in that I'm able to create some images very fast when using the settings in the included workflow. However, after having tried many permutations, I'm unable to get most of the sampler inputs to make any difference. To be specific:
img load error is below.
I'll try this more on a clean comfy install w/o other custom nodes for grins, but that seems unlikely to change the result. Anyone else seeing these sorts of issues?
The expanded size of the tensor (13) must match the existing size (16) at non-singleton dimension 0. Target sizes: [13]. Tensor sizes: [16]
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\execution.py", line 154, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\execution.py", line 84, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\execution.py", line 77, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\custom_nodes\ComfyUI_StreamDiffusion\nodes.py", line 232, in sample
output = model.sample(image).permute(0, 2, 3, 1)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\custom_nodes\ComfyUI_StreamDiffusion\streamdiffusion\wrapper.py", line 332, in sample
image_tensor = self.stream.sample(image)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\custom_nodes\ComfyUI_StreamDiffusion\streamdiffusion\pipeline.py", line 540, in sample
x_0_pred_out = self.predict_x0_batch(x_t_latent)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\custom_nodes\ComfyUI_StreamDiffusion\streamdiffusion\pipeline.py", line 443, in predict_x0_batch
x_0_pred_batch, model_pred = self.unet_step(x_t_latent, t_list)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\custom_nodes\ComfyUI_StreamDiffusion\streamdiffusion\pipeline.py", line 356, in unet_step
model_pred = self.unet(
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\hal\stable-diffusion\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\diffusers\models\unet_2d_condition.py", line 923, in forward
timesteps = timesteps.expand(sample.shape[0])
When I load the workflow both StreamDiffusion_Loader and StreamDiffusion_Sampler nodes are in red, and when I search for StreamDiffusion_Loader to manually replace, that node isn't availble. No obvious error thrown in the terminal.
StreamDiffusion_Sampler can be searched for without issue.
Looking forward to testing this out, thanks for releasing.
Edit: The issue persisted after several restarts, no errors thrown, however after updating another custom node and reloading, the issue seems to have resolved itself. No clues as to what was the cause unfortunately.
step-1: Open this folder:
custom_nodes\ComfyUI_StreamDiffusion,
step-2: type CMD in the address , enter.
step-3: in CMD, type it.
..\..\..\python_embeded\python.exe -s -m pip install diffusers==0.25.0
enter.
step-4: Restart ComfyUI.
Maybe a noob question, but I have tried to import the workflow image as indicated by the nodes are not found by comfyUI, and they appear red. Ho can I install the custom nodes?
Thanks for any help
Hello !
Thank you for making this ComfyUI module.
I would really like to use it, but I have the following errors when trying to :
When using a 1.5 model I have this error (happens when it's on the Sampler step) :
And when i'm using an SDXL model or Turbo, I have this error (happens on the "loading model" step) :
Btw, I also had the problem that the module wouldn't import because of an error with diffusers, it couldn't find LCMsomething in diffusers, I fixed it by running " pip install git+https://github.com/huggingface/diffusers "
Edit : For SDXL I have my answer I guess, it seems it does not support it yet : cumulo-autumn/StreamDiffusion#86
For the 1.5 models, there may be the start of an answer here : cumulo-autumn/StreamDiffusion#62
Edit 2 : Fixed the issue with 1.5 models by modifying the image_utils.py file, just like cvv-student did in the link above :
def denormalize(images: Union[torch.Tensor, np.ndarray]) -> torch.Tensor:
"""
Denormalize an image array to [0,1].
"""
if images.dtype==torch.float16 and images.device.type=="cpu": #added
images = images.float() #added
return (images / 2 + 0.5).clamp(0, 1)
Any idea how I could make this work ?
You could say the issue is solved, but a cleaner way to solve it would probably be a commit in the streamdiffusion original github to solve it for everyone directly.
Thanks in advance.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.