Comments (7)
afeezaziz
commented on May 19, 2024
3
@saketh12 @superadi04 please enable your account to receive monthly support. I will donate on monthly basis so that this can be done/run.
from auto1111sdk.
superadi04
commented on May 19, 2024
Hi @fyepi, this is the next step we are planning to add. @afeezaziz Thank you for the donations, we will set it up and get back to you
from auto1111sdk.
saketh12
commented on May 19, 2024
Hi @afeezaziz I just opened up my Github sponsors account. We appreciate any donations! As requested, we are working on making ControlNet now and expanding the project.
from auto1111sdk.
superadi04
commented on May 19, 2024
Hey, i just added controlnet. Right now it only works with fp32, looking to add fp16 support soon. Also, I would really appreciate any contributors for this issue.
from auto1111sdk.
AlexMKramer
commented on May 19, 2024
Just a note; the instructions for Controlnet are a little confusing and possibly incorrect in some places. This section in the Readme says to use "ControlNetModel(model=" for the control net model, but I am seeing "model_path" as the expected variable.
from auto1111sdk import StableDiffusionPipeline
from auto1111sdk import ControlNetModel
model = ControlNetModel(model="<THE CONTROLNET MODEL FILE NAME (WITHOUT EXTENSION)>",
image="<PATH TO IMAGE>")
pipe = StableDiffusionPipeline("<Path to your local safetensors or checkpoint file>", controlnet=model)
prompt = "a picture of a brown dog"
output = pipe.generate_txt2img(prompt = prompt, height = 1024, width = 768, steps = 10)
output[0].save("image.png")The Gitbook also is inconsistent showing the instruction above, but shows "ControlNet(model=" in the example.
Not sure what the proper way to set the model is either way, as setting it with "model=" says unexpected argument, "model_path" doesn't seem to set the model properly, and "ControlNet("model=" is unrecognized.
from auto1111sdk.
superadi04
commented on May 19, 2024
hey could you send me the code you are running exaclty
from auto1111sdk.
AlexMKramer
commented on May 19, 2024
Here is an example of the code I am running:
import os
import time
from auto1111sdk import StableDiffusionPipeline, ControlNetModel
def controlnet(prompt="a man eating an ice cream cone",
negative_prompt="",
model_path="models/checkpoints/epicPhotogasm.safetensors",
num_images=4,
height=512,
width=512,
steps=10,
cfg_scale=7,
sampler_name="DPM++ 2M Karras",
clip_skip=2,
controlnet_name="controlnet_depth"
):
print(model_path)
# If a controlnet is specified, run the image through the controlnet first
controlnet_path = os.path.join(os.getcwd(), "models/controlnets/" + controlnet_name)
print(controlnet_path)
controlnet_model = ControlNetModel(controlnet_path, image="attached_image.png")
pipe = StableDiffusionPipeline(model_path, controlnet=controlnet_model, clip_skip=clip_skip)
print(f"Using {controlnet_name} to process the image")
output = pipe.generate_txt2img(prompt=prompt,
negative_prompt=negative_prompt,
num_images=num_images,
height=height,
width=width,
steps=steps,
cfg_scale=cfg_scale,
sampler_name=sampler_name,
seed=-1
)
file_paths = []
for i in range(num_images):
# Save the output to a file in the output folder using the time and date as the file name
file_name = f"ImageGenBot-{time.strftime('%Y-%m-%d-%H-%M-%S')}-{i}.png"
save_path = os.path.join(os.getcwd(), "output/")
output[i].save(f"{save_path}{file_name}")
print(f"Saved {file_name}")
file_paths.append(f"{save_path}{file_name}")
pipe.set_vae(None)
del pipe
print("Finished generating controlnet images.")
controlnet()
print("Done!")
And here is the output:
(base) alexk@AlexHomeServer:~/Desktop/ImageGenBot$ venv/bin/python test.py
models/checkpoints/epicPhotogasm.safetensors
/home/alexk/Desktop/ImageGenBot/models/controlnets/controlnet_depth
None
2024-03-20 07:56:59,104 - ControlNet - INFO - ControlNet v1.1.440
ControlNet preprocessor location: /home/alexk/Desktop/ImageGenBot/venv/lib/python3.11/site-packages/auto1111sdk/extensions/controlnet/annotator/downloads
2024-03-20 07:57:00,435 - ControlNet - INFO - ControlNet v1.1.440
./
None
Applying attention optimization: Doggettx... done.
Using controlnet_depth to process the image
2024-03-20 07:57:02,898 - ControlNet - INFO - unit_separate = False, style_align = False
*** Error running process: /home/alexk/Desktop/ImageGenBot/venv/lib/python3.11/site-packages/auto1111sdk/extensions/controlnet/scripts/controlnet.py
Traceback (most recent call last):
File "/home/alexk/Desktop/ImageGenBot/venv/lib/python3.11/site-packages/auto1111sdk/modules/scripts.py", line 715, in process
script.process(p, *script_args)
File "/home/alexk/Desktop/ImageGenBot/venv/lib/python3.11/site-packages/auto1111sdk/extensions/controlnet/scripts/controlnet.py", line 1191, in process
self.controlnet_hack(p)
File "/home/alexk/Desktop/ImageGenBot/venv/lib/python3.11/site-packages/auto1111sdk/extensions/controlnet/scripts/controlnet.py", line 1176, in controlnet_hack
self.controlnet_main_entry(p)
File "/home/alexk/Desktop/ImageGenBot/venv/lib/python3.11/site-packages/auto1111sdk/extensions/controlnet/scripts/controlnet.py", line 941, in controlnet_main_entry
model_net, control_model_type = Script.load_control_model(p, unet, unit.model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alexk/Desktop/ImageGenBot/venv/lib/python3.11/site-packages/auto1111sdk/extensions/controlnet/scripts/controlnet.py", line 408, in load_control_model
control_model = Script.build_control_model(p, unet, model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alexk/Desktop/ImageGenBot/venv/lib/python3.11/site-packages/auto1111sdk/extensions/controlnet/scripts/controlnet.py", line 427, in build_control_model
raise RuntimeError(f"model not found: {model}")
RuntimeError: model not found: None
---
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01<00:00, 6.52it/s]
Total progress: 10it [00:01, 8.28it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01<00:00, 8.19it/s]
Total progress: 10it [00:01, 8.46it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01<00:00, 8.16it/s]
Total progress: 10it [00:01, 8.45it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01<00:00, 8.22it/s]
Total progress: 10it [00:01, 8.44it/s]
Saved ImageGenBot-2024-03-20-07-57-08-0.png
Saved ImageGenBot-2024-03-20-07-57-08-1.png
Saved ImageGenBot-2024-03-20-07-57-08-2.png
Saved ImageGenBot-2024-03-20-07-57-09-3.png
Finished generating controlnet images.
Done!
If it is something I am doing incorrectly, let me know!
from auto1111sdk.
Related Issues (19)
- Unity integration (GUI support for VR and Desktop)
- Is there a way to load a VAE? HOT 2
- feature request: seed travel / interpolation HOT 3
- Possible pytorch-cuda compatibility issue on windows. HOT 4
- Use https://github.com/lllyasviel/stable-diffusion-webui-forge as backend? HOT 2
- API Key support for civit_download HOT 3
- Switching models within the same script HOT 3
- Please use semver and add a CHANGELOG HOT 1
- How to get random seed value? HOT 2
- the gpu vram stays same after certain generations HOT 6
- Will you be supporting video? HOT 1
- Does it support prompt weighting? HOT 1
- Model creating completely different outputs between Auto1111SDK and A1111
- Does it support Controlnet reference only?
- Will LORA support be enabled in near future?
- Dual Gpu support?
- Can't install
- NO "fill", "original", "latent noise" option! HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from auto1111sdk.