bes-dev / stable_diffusion.openvino Goto Github PK

View Code? Open in Web Editor NEW

1.5K 38.0 208.0 2.62 MB

License: Apache License 2.0

Python 100.00%

stable_diffusion.openvino's Introduction

stable_diffusion.openvino

Implementation of Text-To-Image generation using Stable Diffusion on Intel CPU or GPU.

Requirements

Linux, Windows, MacOS
Python <= 3.9.0
CPU or GPU compatible with OpenVINO.

Install requirements

Set up and update PIP to the highest version
Install OpenVINO™ Development Tools 2022.3.0 release with PyPI
Download requirements

python -m pip install --upgrade pip
pip install openvino-dev[onnx,pytorch]==2022.3.0
pip install -r requirements.txt

Generate image from text description

usage: demo.py [-h] [--model MODEL] [--device DEVICE] [--seed SEED] [--beta-start BETA_START] [--beta-end BETA_END] [--beta-schedule BETA_SCHEDULE]
               [--num-inference-steps NUM_INFERENCE_STEPS] [--guidance-scale GUIDANCE_SCALE] [--eta ETA] [--tokenizer TOKENIZER] [--prompt PROMPT] [--params-from PARAMS_FROM]
               [--init-image INIT_IMAGE] [--strength STRENGTH] [--mask MASK] [--output OUTPUT]

optional arguments:
  -h, --help            show this help message and exit
  --model MODEL         model name
  --device DEVICE       inference device [CPU, GPU]
  --seed SEED           random seed for generating consistent images per prompt
  --beta-start BETA_START
                        LMSDiscreteScheduler::beta_start
  --beta-end BETA_END   LMSDiscreteScheduler::beta_end
  --beta-schedule BETA_SCHEDULE
                        LMSDiscreteScheduler::beta_schedule
  --num-inference-steps NUM_INFERENCE_STEPS
                        num inference steps
  --guidance-scale GUIDANCE_SCALE
                        guidance scale
  --eta ETA             eta
  --tokenizer TOKENIZER
                        tokenizer
  --prompt PROMPT       prompt
  --params-from PARAMS_FROM
                        Extract parameters from a previously generated image.
  --init-image INIT_IMAGE
                        path to initial image
  --strength STRENGTH   how strong the initial image should be noised [0.0, 1.0]
  --mask MASK           mask of the region to inpaint on the initial image
  --output OUTPUT       output image name

Examples

Example Text-To-Image

python demo.py --prompt "Street-art painting of Emilia Clarke in style of Banksy, photorealism"

Example Image-To-Image

python demo.py --prompt "Photo of Emilia Clarke with a bright red hair" --init-image ./data/input.png --strength 0.5

Example Inpainting

python demo.py --prompt "Photo of Emilia Clarke with a bright red hair" --init-image ./data/input.png --mask ./data/mask.png --strength 0.5

Performance

CPU	Time per iter	Total time
AMD Ryzen 7 4800H	4.8 s/it	2.58 min
AMD Ryzen Threadripper 1900X	5.34 s/it	2.58 min
Intel(R) Core(TM) i7-4790K @ 4.00GHz	10.1 s/it	5.39 min
Intel(R) Core(TM) i5-8279U	7.4 s/it	3.59 min
Intel(R) Core(TM) i5-8569U @ 2.8GHz (MBP13-2019)	6.17 s/it	3.23 min
Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	7.4 s/it	3.59 min
Intel(R) Core(TM) i7-11800H @ 2.30GHz (16 threads)	2.9 s/it	1.54 min
Intel(R) Core(TM) i7-1280P @ 1.80GHz (6P/8E)	5.45 s/it	2.55 min
Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz	1 s/it	33 s
Intel Arc A770M	6.64 it/s	7.53 s

Acknowledgements

Original implementation of Stable Diffusion: https://github.com/CompVis/stable-diffusion
diffusers library: https://github.com/huggingface/diffusers

Disclaimer

The authors are not responsible for the content generated using this project. Please, don't use this project to produce illegal, harmful, offensive etc. content.

stable_diffusion.openvino's People

Contributors

Stargazers

Watchers

Forkers

thr3a marcus-arcadius mo-adi morkovka1337 sondro dervish54 remcova rabidcopy zivzone techthiyanes amrrs mikhael-danilov meganetaaan pnrajan rquiroga7 jamesthesnake respondgaurav therealbarenziah abul22 06opotehb belzecue dranov jbtiv neuw84 falahgs highbridgedragon alathavbo khm laplacekorea metavai richardsonjf theolivenbaum eafyounian atannir duaneking tzygeneral zvrr 0tao kadirseker00 jytime honette stepheodhiambo2000 pvrajput tfab87 impca breadbrowser 0xdigest elleryw davidson00000 alexkoff88 ssenthilanand rakarcloud justcaptcha miraium ecliptik scaamanho codrineugeniu commanderz martinnowak koji blueskydetector maddyonline nnuudev ojeiofj timwi dominico120 afiqmuzaffar tungvuthanh sitresearch reachea jkrafczyk pihtong cedspam cdingram wrongsyntax kamjin1996 hawkclaws senthilnayagam ebibibi matrixy reiwa monkeywidget jcolicchio bankainojutsu ma5onic jwhy89 azrael511 changtraixuqang97 kephale auraadeep klopez89 golv1974 sgwd jaedukseo python-repository-hub drake53 dagrambraut icethunder matthewk607 nitin-nizhawan

stable_diffusion.openvino's Issues

[not an issue] How long does it takes on average to generate an image using the CPU?

Hi, I'd like to setup this for a friend on their laptop so I was wondering how long does it take to generate an image using an average CPU?
Are we talking about a couple of minutes, or will it take way longer than that?
Just trying to get an idea to understand if it's worth it or not.

Not able to duplicate images

I was able to generate some great starter images, but when I try to recreate them in other tools I'm not able to.
For example, if I run
python3.9 demo.py --seed "18012747" --num-inference-steps "100" --guidance-scale "7.5" --prompt "A Photograph of lightning in a bottle" --output "/tmp/output.png"
then I get the following image,

But when I try the same prompt and settings on this SD tool, https://github.com/sd-webui/stable-diffusion-webui , I get the following image,

Can anyone explain the difference? I thought both were SD 1.4. Same prompt, seed, guidance_scale, steps. I tried all the samplers available to the sd-webui versions in case that was it.

Model Name and how to replace it?

I know the location of the model but not the name. How do I replace it with a finetuned model e.g. waifu diffusion?

Suggestion: Add --seed for generating consistent images and testing the effect of hyper-params

For example, the following implementation will allow a seed to be specified:


def main(args):
    if args.seed is not None:
        np.random.seed(args.seed)
    scheduler = LMSDiscreteScheduler(
        beta_start=args.beta_start,
        beta_end=args.beta_end,
        beta_schedule=args.beta_schedule,
        tensor_format="np"
    )
    stable_diffusion = StableDiffusion(
        model = args.model,
        scheduler = scheduler,
        tokenizer = args.tokenizer
    )
    image = stable_diffusion(
        prompt = args.prompt,
        num_inference_steps = args.num_inference_steps,
        guidance_scale = args.guidance_scale,
        eta = args.eta
    )
    cv2.imwrite(args.output, image)


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    # pipeline configure
    parser.add_argument("--model", type=str, default="bes-dev/stable-diffusion-v1-4-openvino", help="model name")
    # scheduler params
    parser.add_argument("--beta-start", type=float, default=0.00085, help="LMSDiscreteScheduler::beta_start")
    parser.add_argument("--beta-end", type=float, default=0.012, help="LMSDiscreteScheduler::beta_end")
    parser.add_argument("--beta-schedule", type=str, default="scaled_linear", help="LMSDiscreteScheduler::beta_schedule")
    # diffusion params
    parser.add_argument("--num-inference-steps", type=int, default=32, help="num inference steps")
    parser.add_argument("--guidance-scale", type=float, default=7.5, help="guidance scale")
    parser.add_argument("--eta", type=float, default=0.0, help="eta")
    # tokenizer
    parser.add_argument("--tokenizer", type=str, default="openai/clip-vit-large-patch14", help="tokenizer")
    # prompt
    parser.add_argument("--prompt", type=str, default="Street-art painting of Emilia Clarke in style of Banksy, photorealism", help="prompt")
    # seed
    parser.add_argument("--seed", type=int, default=None, help="Random seed for generating consistent images per prompt")
    # output name
    parser.add_argument("--output", type=str, default="output.png", help="output image name")
    args = parser.parse_args()
    main(args)

Which allows for comparing 10 vs. 32 steps per images with the same prompt:

Is there someone meet this Error?

Windows 10 / Intel I7-6700 / Python 3.8.5 / pip 20.3.3

(ldc) D:\py2\stable_diffusion.openvino>python demo.py --prompt "Photo of Emilia Clarke with a bright red hair" --init-image ./data/input.png --mask ./data/mask.png --strength 0.5
Traceback (most recent call last):
  File "demo.py", line 74, in <module>
    main(args)
  File "demo.py", line 31, in main
    engine = StableDiffusionEngine(
  File "D:\py2\stable_diffusion.openvino\stable_diffusion_engine.py", line 37, in __init__
    self._unet = self.core.read_model(
RuntimeError: Check 'PartialShape::broadcast_merge_into(pshape, node->get_input_partial_shape(i), autob)' failed at C:\j\workspace\private-ci\ie\build-windows-vs2019@3\b\repos\openvino\src\core\src\op\util\elementwise_args.cpp:30:
While validating node 'v1::Multiply Multiply_16273 (onnx::Mul_1660[0]:f32{2,32,32}, Constant_150330[0]:f32{1,1,640}) -> ()' with friendly_name 'Multiply_16273':
Argument shapes are inconsistent.

PS: pip list

(ldc) D:\py2\stable_diffusion.openvino>pip list
Package Version Location

absl-py 1.2.0
addict 2.4.0
aiohttp 3.8.1
aiosignal 1.2.0
albumentations 0.4.3
altair 4.2.0
antlr4-python3-runtime 4.8
async-timeout 4.0.2
attrs 22.1.0
backports.zoneinfo 0.2.1
blinker 1.5
brotlipy 0.7.0
cachetools 5.2.0
certifi 2022.6.15
cffi 1.15.1
charset-normalizer 2.0.4
click 8.1.3
clip 1.0 d:\py2\stable-diffusion-cpu\src\clip
colorama 0.4.5
commonmark 0.9.1
cryptography 37.0.1
decorator 5.1.1
defusedxml 0.7.1
diffusers 0.2.4
editdistance 0.6.0
einops 0.3.0
entrypoints 0.4
fast-ctc-decode 0.3.2
filelock 3.8.0
flatbuffers 2.0.7
frozenlist 1.3.1
fsspec 2022.8.2
ftfy 6.1.1
future 0.18.2
gitdb 4.0.9
GitPython 3.1.27
google-auth 2.11.0
google-auth-oauthlib 0.4.6
grpcio 1.48.1
huggingface-hub 0.9.0
hyperopt 0.1.2
idna 3.3
imagecodecs 2022.2.22
imageio 2.9.0
imageio-ffmpeg 0.4.2
imgaug 0.2.6
importlib-metadata 4.12.0
importlib-resources 5.9.0
invisible-watermark 0.1.5
Jinja2 3.1.2
joblib 1.1.0
jsonschema 4.15.0
jstyleson 0.0.2
kornia 0.6.0
latent-diffusion 0.0.1 d:\py2\stable-diffusion-cpu
lmdb 1.3.0
Markdown 3.4.1
MarkupSafe 2.1.1
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
multidict 6.0.2
networkx 2.8.6
nibabel 4.0.2
nltk 3.7
numpy 1.19.5
oauthlib 3.2.0
omegaconf 2.1.1
onnx 1.12.0
onnxruntime 1.10.0
opencv-python 4.5.5.64
opencv-python-headless 4.6.0.66
openvino 2022.1.0
openvino-dev 2022.1.0
openvino-telemetry 2022.1.1
packaging 21.3
pandas 1.1.5
parasail 1.2.4
Pillow 9.2.0
pip 20.3.3
pkgutil-resolve-name 1.3.10
progress 1.6
protobuf 3.19.4
pudb 2019.2
py-cpuinfo 8.0.0
pyarrow 9.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pyclipper 1.3.0.post3
pycparser 2.21
pydeck 0.8.0b1
pyDeprecate 0.3.1
pydicom 2.3.0
Pygments 2.13.0
pymongo 4.2.0
Pympler 1.0.1
pyOpenSSL 22.0.0
pyparsing 3.0.9
pyrsistent 0.18.1
PySocks 1.7.1
python-dateutil 2.8.2
pytorch-lightning 1.4.2
pytz 2022.2.1
pytz-deprecation-shim 0.1.0.post0
PyWavelets 1.3.0
PyYAML 6.0
rawpy 0.17.2
regex 2022.8.17
requests 2.28.1
requests-oauthlib 1.3.1
rich 12.5.1
rsa 4.9
sacremoses 0.0.53
scikit-image 0.19.3
scikit-learn 0.24.2
scipy 1.5.4
semver 2.13.0
sentencepiece 0.1.97
setuptools 63.4.1
Shapely 1.8.4
six 1.16.0
smmap 5.0.0
streamlit 1.12.0
taming-transformers 0.0.1 d:\py2\stable-diffusion-cpu\src\taming-transformers
tensorboard 2.10.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
test-tube 0.7.5
texttable 1.6.3
threadpoolctl 3.1.0
tifffile 2021.11.2
tokenizers 0.10.3
toml 0.10.2
toolz 0.12.0
torch 1.11.0
torch-fidelity 0.3.0
torchmetrics 0.6.0
torchvision 0.12.0
tornado 6.2
tqdm 4.64.0
transformers 4.16.2
typing-extensions 4.3.0
tzdata 2022.2
tzlocal 4.2
urllib3 1.26.11
urwid 2.1.2
validators 0.20.0
watchdog 2.1.9
wcwidth 0.2.5
Werkzeug 2.2.2
wheel 0.37.1
win-inet-pton 1.1.0
wincertstore 0.2
yamlloader 1.1.0
yarl 1.8.1
zipp 3.8.1

Force stable_diffusion to run in x86-64 mode on Docker Mac-M1 (and other OS's running on Arm)

See the suggestion here:
#15 (comment)

Add --platform=linux/amd64 to the docker file like so:

FROM --platform=linux/amd64 python:3.9.9-bullseye

Any possibility for different resolutions

The current model is restricted to 512x512 pixels. Can you please give a tutorial to customize openvino models with the resolutions of choice? For example, 768x512 pixels?

General thoughts

Just wanted to put this here for anyone who had some headaches with this and all the other versions of stable diffusion.

If you specify the output file, you need to add the extension as well. So "outputs/sample.png" If you don't add the extension it will fully generate your image, then fail at the end with no clear solution.
I'm running a Ryzen 5 3600 and getting ~4.48t/s with 16GB of fairly cheap ram, so it could get even better

Location of model and general question about size

Hi!

Thanks for the great project, it's the only working for me at the moment (amd, linux and RX560) and a huge step on putting Stable Diffusion to the masses! It works wonderfully.

My question is, where is the model located? It's the same as the 4gb version on huggingface? I'm on a small SSD I want to keep track of my huge files hah.

Also, could I change the size of the images to something bigger than 512x512?

Thanks! Sorry I'm a newb on this.

error: metadata-generation failed

C:\Users\Henrich Viviers\Documents\stable_diffusion.openvino-master>pip install -r requirements.txt
Collecting numpy==1.19.5
Downloading numpy-1.19.5.zip (7.3 MB)
---------------------------------------- 7.3/7.3 MB 4.2 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [267 lines of output]
setup.py:67: RuntimeWarning: NumPy 1.19.5 may not yet support Python 3.10.
warnings.warn(
Running from numpy source directory.
setup.py:480: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
run_build = parse_setuppy_commands()
Processing numpy/random_bounded_integers.pxd.in
Processing numpy/random\bit_generator.pyx
Processing numpy/random\mtrand.pyx
Processing numpy/random_bounded_integers.pyx.in
Processing numpy/random_common.pyx
Processing numpy/random_generator.pyx
Processing numpy/random_mt19937.pyx
Processing numpy/random_pcg64.pyx
Processing numpy/random_philox.pyx
Processing numpy/random_sfc64.pyx
Cythonizing sources
blas_opt_info:
blas_mkl_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
libraries mkl_rt not found in ['C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib', 'C:\', 'C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs']
NOT AVAILABLE

  blis_info:
    libraries blis not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  openblas_info:
    libraries openblas not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
  get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
  customize GnuFCompiler
  Could not locate executable g77
  Could not locate executable f77
  customize IntelVisualFCompiler
  Could not locate executable ifort
  Could not locate executable ifl
  customize AbsoftFCompiler
  Could not locate executable f90
  customize CompaqVisualFCompiler
  Could not locate executable DF
  customize IntelItaniumVisualFCompiler
  Could not locate executable efl
  customize Gnu95FCompiler
  Could not locate executable gfortran
  Could not locate executable f95
  customize G95FCompiler
  Could not locate executable g95
  customize IntelEM64VisualFCompiler
  customize IntelEM64TFCompiler
  Could not locate executable efort
  Could not locate executable efc
  customize PGroupFlangCompiler
  Could not locate executable flang
  don't know how to compile Fortran code on platform 'nt'
    NOT AVAILABLE

  atlas_3_10_blas_threads_info:
  Setting PTATLAS=ATLAS
    libraries tatlas not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  atlas_3_10_blas_info:
    libraries satlas not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  atlas_blas_threads_info:
  Setting PTATLAS=ATLAS
    libraries ptf77blas,ptcblas,atlas not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  atlas_blas_info:
    libraries f77blas,cblas,atlas not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  accelerate_info:
    NOT AVAILABLE

  C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\system_info.py:1914: UserWarning:
      Optimized (vendor) Blas libraries are not found.
      Falls back to netlib Blas library which has worse performance.
      A better performance should be easily gained by switching
      Blas library.
    if self._calc_info(blas):
  blas_info:
    libraries blas not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\system_info.py:1914: UserWarning:
      Blas (http://www.netlib.org/blas/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [blas]) or by setting
      the BLAS environment variable.
    if self._calc_info(blas):
  blas_src_info:
    NOT AVAILABLE

  C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\system_info.py:1914: UserWarning:
      Blas (http://www.netlib.org/blas/) sources not found.
      Directories to search for the sources can be specified in the
      numpy/distutils/site.cfg file (section [blas_src]) or by setting
      the BLAS_SRC environment variable.
    if self._calc_info(blas):
    NOT AVAILABLE

  non-existing path in 'numpy\\distutils': 'site.cfg'
  lapack_opt_info:
  lapack_mkl_info:
    libraries mkl_rt not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  openblas_lapack_info:
    libraries openblas not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  openblas_clapack_info:
    libraries openblas,lapack not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  flame_info:
    libraries flame not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  atlas_3_10_threads_info:
  Setting PTATLAS=ATLAS
    libraries lapack_atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib
    libraries tatlas,tatlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib
    libraries lapack_atlas not found in C:\
    libraries tatlas,tatlas not found in C:\
    libraries lapack_atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs
    libraries tatlas,tatlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs
  <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
    NOT AVAILABLE

  atlas_3_10_info:
    libraries lapack_atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib
    libraries satlas,satlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib
    libraries lapack_atlas not found in C:\
    libraries satlas,satlas not found in C:\
    libraries lapack_atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs
    libraries satlas,satlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs
  <class 'numpy.distutils.system_info.atlas_3_10_info'>
    NOT AVAILABLE

  atlas_threads_info:
  Setting PTATLAS=ATLAS
    libraries lapack_atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib
    libraries ptf77blas,ptcblas,atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib
    libraries lapack_atlas not found in C:\
    libraries ptf77blas,ptcblas,atlas not found in C:\
    libraries lapack_atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs
    libraries ptf77blas,ptcblas,atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs
  <class 'numpy.distutils.system_info.atlas_threads_info'>
    NOT AVAILABLE

  atlas_info:
    libraries lapack_atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib
    libraries f77blas,cblas,atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib
    libraries lapack_atlas not found in C:\
    libraries f77blas,cblas,atlas not found in C:\
    libraries lapack_atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs
    libraries f77blas,cblas,atlas not found in C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\libs      <class 'numpy.distutils.system_info.atlas_info'>
    NOT AVAILABLE

  lapack_info:
    libraries lapack not found in ['C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\lib', 'C:\\', 'C:\\Users\\Henrich Viviers\\AppData\\Local\\Programs\\Python\\Python310\\libs']
    NOT AVAILABLE

  C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\system_info.py:1748: UserWarning:
      Lapack (http://www.netlib.org/lapack/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [lapack]) or by setting
      the LAPACK environment variable.
    return getattr(self, '_calc_info_{}'.format(name))()
  lapack_src_info:
    NOT AVAILABLE

  C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\system_info.py:1748: UserWarning:
      Lapack (http://www.netlib.org/lapack/) sources not found.
      Directories to search for the sources can be specified in the
      numpy/distutils/site.cfg file (section [lapack_src]) or by setting
      the LAPACK_SRC environment variable.
    return getattr(self, '_calc_info_{}'.format(name))()
    NOT AVAILABLE

  numpy_linalg_lapack_lite:
    FOUND:
      language = c
      define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]

  C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\dist.py:275: UserWarning: Unknown distribution option: 'define_macros'
    warnings.warn(msg)
  running dist_info
  running build_src
  build_src
  building py_modules sources
  creating build
  creating build\src.win-amd64-3.10
  creating build\src.win-amd64-3.10\numpy
  creating build\src.win-amd64-3.10\numpy\distutils
  building library "npymath" sources
  Traceback (most recent call last):
    File "C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 363, in <module>
      main()
    File "C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Users\Henrich Viviers\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 164, in prepare_metadata_for_build_wheel
      return hook(metadata_directory, config_settings)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\build_meta.py", line 157, in prepare_metadata_for_build_wheel
      self.run_setup()
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\build_meta.py", line 248, in run_setup
      super(_BuildMetaLegacyBackend,
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 508, in <module>
      setup_package()
    File "setup.py", line 500, in setup_package
      setup(**metadata)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\core.py", line 169, in setup
      return old_setup(**new_attr)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\__init__.py", line 165, in setup
      return distutils.core.setup(**attrs)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 148, in setup
      dist.run_commands()
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 967, in run_commands
      self.run_command(cmd)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
      cmd_obj.run()
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\command\dist_info.py", line 31, in run
      egg_info.run()
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\egg_info.py", line 24, in run
      self.run_command("build_src")
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
      cmd_obj.run()
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\build_src.py", line 144, in run
      self.build_sources()
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\build_src.py", line 155, in build_sources
      self.build_library_sources(*libname_info)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\build_src.py", line 288, in build_library_sources
      sources = self.generate_sources(sources, (lib_name, build_info))
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\build_src.py", line 378, in generate_sources
      source = func(extension, build_dir)
    File "numpy\core\setup.py", line 658, in get_mathlib_info
      st = config_cmd.try_link('int main(void) { return 0;}')
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 243, in try_link
      self._link(body, headers, include_dirs,
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\config.py", line 162, in _link
      return self._wrap_method(old_config._link, lang,
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\config.py", line 96, in _wrap_method
      ret = mth(*((self,)+args))
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 137, in _link
      (src, obj) = self._compile(body, headers, include_dirs, lang)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\config.py", line 105, in _compile
      src, obj = self._wrap_method(old_config._compile, lang,
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\command\config.py", line 96, in _wrap_method
      ret = mth(*((self,)+args))
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 132, in _compile
      self.compiler.compile([src], include_dirs=include_dirs)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 401, in compile
      self.spawn(args)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-build-env-gft7ttbj\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 505, in spawn
      return super().spawn(cmd, env=env)
    File "C:\Users\Henrich Viviers\AppData\Local\Temp\pip-install-hyv2jnto\numpy_7ba62efa9e2c48f195c13f2e715ff570\numpy\distutils\ccompiler.py", line 90, in <lambda>
      m = lambda self, *args, **kw: func(self, *args, **kw)
  TypeError: CCompiler_spawn() got an unexpected keyword argument 'env'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Only 4 threads seem to be used on an 8 thread machine.

Hi! This is a really cool piece of work, seems to run approx 2x faster than a native Torch CPU implementation. I did notice that it only uses 4 of the 8 threads on my machine though. I'm new to openvino; is there a way to configure how many threads are used?

Missing requirements for streamlit

Creating a docker image and running streamlit with demo_web.py gives the following error immediately upon entering the web app from a browser:

$ docker run -p 8501:8501 sdopenvino:latest

  You can now view your Streamlit app in your browser.

  Network URL: http://172.17.0.2:8501
  External URL: http://80.208.65.21:8501

2022-09-04 17:46:11.731 Uncaught app exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 556, in _run_script
    exec(code, module.__dict__)
  File "/src/demo_web.py", line 6, in <module>
    from streamlit_drawable_canvas import st_canvas
ModuleNotFoundError: No module named 'streamlit_drawable_canvas'

I fixed this by running a pip install of the streamlit_drawable_canvas module. I guess this module should be part of requirements.txt?

Provide Dockerfile and published Docker image

It would help to have a Dockerfile and a published Docker image. This will allow convenient usage without having to install the requirements. Providing a Docker Compose file will also help, such as to configure the number of CPUs as per issue #10.

The Compose file could map the directories into which data is downloaded, and where the file is written out, so they're writable. As noted in issue #12, it would be great if the Docker image is fully self sufficient, so it can work entirely offline.

As a limited example, but probably not a perfect one, see https://github.com/AbdBarho/stable-diffusion-webui-docker , although I am not requesting a webUI if that's complicated.

404 Client Error: Entry Not Found for url: https://hugging...vae.xml

Hi, have some problems, any hint?

❯ python3.9 -m pip freeze
alembic==1.7.4
appdirs==1.4.4
argcomplete==1.12.2
argon2-cffi==20.1.0
asn1crypto==1.4.0
async-generator==1.10
attrs==20.3.0
Authlib==0.15.4
Babel==2.9.0
bcrypt==3.2.0
Beaker==1.11.0
beautifulsoup4==4.9.3
bidict==0.21.3
black==21.5b2
bleach==3.3.0
blinker==1.4
blis==0.7.8
blivet==3.3.2
Bottleneck==1.3.2
Brotli==1.0.9
bs4==0.0.1
cached-property==1.5.2
cachetools==4.2.1
catalogue==2.0.8
certifi==2020.12.5
cffi==1.14.5
chardet==3.0.4
click==8.1.3
coverage==5.4
cryptography==3.4.6
cymem==2.0.6
dearpygui==1.0.0
diffusers==0.2.4
distlib==0.3.1
dnspython==1.16.0
docker==4.4.2
docker-compose==1.28.2
email-validator==1.1.3
eventlet==0.31.0
filelock==3.0.12
Flask==1.1.4
Flask-BabelEx==0.9.4
Flask-Compress==1.10.1
Flask-Gravatar==0.5.0
Flask-Login==0.5.0
Flask-Mail==0.9.1
Flask-Migrate==2.7.0
Flask-Paranoid==0.2.0
Flask-Principal==0.4.0
Flask-Security-Too==4.1.2
Flask-SocketIO==5.1.1
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.15.1
ftfy==6.1.1
google==3.0.0
google-api-core==1.26.0
google-api-python-client==1.12.8
google-auth==1.27.0
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.2
googleapis-common-protos==1.53.0
greenlet==1.1.2
httpagentparser==1.9.1
httplib2==0.19.0
huggingface-hub==0.9.1
humanize==3.2.0
idna==2.10
imageio==2.9.0
img2pdf==0.4.0
importlib-metadata==4.12.0
iniparse==0.5
invoke==1.5.0
ipykernel==5.4.3
ipython==7.20.0
itsdangerous==1.1.0
jedi==0.18.0
Jinja2==3.1.2
joblib==1.0.1
jupyter-client==6.1.11
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
langcodes==3.3.0
ldap3==2.9.1
lit==0.11.1
lxml==4.6.2
Mako==1.1.4
MarkupSafe==2.1.1
mistune==0.8.4
mock==4.0.3
murmurhash==1.0.8
mypy-extensions==0.4.3
natsort==7.1.1
nbclient==0.5.2
nbconvert==6.0.7
nest-asyncio==1.5.1
netifaces==0.10.9
NeuroTools==0.3.1
notebook==6.2.0
numexpr==2.7.2
numpy==1.19.5
oauthlib==3.1.0
opencv-python==4.5.5.64
openvino==2022.1.0
ordered-set==4.0.2
packaging==20.9
pandas==1.2.2
parso==0.8.1
passlib==1.7.4
Paste==3.5.0
pathspec==0.8.1
pathy==0.6.2
pbr==5.5.1
pgadmin4==5.7
pid==3.0.4
pikepdf==2.9.2
Pillow==8.1.0
pipenv==2020.11.15
powerline-mem-segment==2.4.1
preshed==3.0.7
productmd==1.31
prometheus-client==0.9.0
prompt-toolkit==3.0.18
protobuf==3.15.3
psutil==5.8.0
psycopg2-binary==2.8.6
ptpython==3.0.17
pycairo==1.20.0
pydantic==1.9.2
pygame==2.0.1
Pygments==2.8.0
PyHamcrest==2.0.2
pykickstart==3.32
PyNaCl==1.4.0
pyparsing==3.0.9
PyQt5==5.15.2
PyQt5-sip==12.8.1
pyrsistent==0.17.3
python-dotenv==0.15.0
python-engineio==4.2.1
python-socketio==5.4.1
python-xlib==0.29
pytoml==0.1.21
pytz==2021.1
PyYAML==6.0
QScintilla==2.11.6
regex==2021.4.4
requests==2.25.1
requests-file==1.5.1
requests-oauthlib==1.3.0
rsa==4.7.2
sacremoses==0.0.53
scipy==1.9.0
scour==0.38.2
selenium==3.141.0
Send2Trash==1.5.0
setproctitle==1.2.2
simplejson==3.17.5
six==1.16.0
smart-open==5.2.1
soupsieve==2.2
spacy==3.4.1
spacy-legacy==3.0.10
spacy-loggers==1.0.3
speaklater==1.3
speaklater3==1.4
SQLAlchemy==1.3.24
sqlparse==0.4.2
srsly==2.4.4
sshtunnel==0.4.0
tables==3.6.1
Tempita==0.5.2
termcolor==1.1.0
terminado==0.9.2
texttable==1.6.3
thinc==8.1.0
tokenizers==0.12.1
toml==0.10.2
torch==1.12.1
tornado==6.1
tqdm==4.64.0
transformers==4.16.2
typer==0.4.2
typing_extensions==4.3.0
ua-parser==0.10.0
uritemplate==3.0.1
urllib3==1.26.3
user-agents==2.2.0
virtualenv==20.4.2
virtualenv-clone==0.5.4
w3lib==1.22.0
wasabi==0.10.1
wcwidth==0.2.5
websocket-client==0.57.0
Werkzeug==1.0.1
WTForms==2.3.3
wxPython==4.1.1
xlrd==2.0.1
xlwt==1.3.0
yapf==0.30.0
yaspin==2.0.0
zipp==3.8.1
zope.event==4.5.0
zope.interface==5.2.0

❯ python3.9 stable_diffusion.py --prompt "Street-art painting of Emilia Clarke in style of Banksy, photorealism"

Traceback (most recent call last):
File "/home/zm/.local/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py", line 131, in _raise_for_status
response.raise_for_status()
File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/bes-dev/stable-diffusion-v1-4-openvino/resolve/main/vae.xml

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/zm/Downloads/SOFT/StableDiffusion/stable_diffusion.py", line 179, in
main(args)
File "/home/zm/Downloads/SOFT/StableDiffusion/stable_diffusion.py", line 144, in main
stable_diffusion = StableDiffusion(
File "/home/zm/Downloads/SOFT/StableDiffusion/stable_diffusion.py", line 45, in init
hf_hub_download(repo_id=model, filename="vae.xml"),
File "/home/zm/.local/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 1099, in hf_hub_download
_raise_for_status(r)
File "/home/zm/.local/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py", line 155, in _raise_for_status
raise e
huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error: Entry Not Found for url: https://huggingface.co/bes-dev/stable-diffusion-v1-4-openvino/resolve/main/vae.xml. (Request ID: 6a3sLwi7XxwZqLdYDcXpy)

Inpainting stops at 26 iterations

Hello,

I set an --num-inference-steps argument, and gave it a value of 50. However, after 26 steps, the app called it quits and finalized the image. The image looks surprisingly great for having only done 26 steps, but I would have liked it if it did go to 50 instead like I instructed it to.

Any ideas on why this could be happening?

Thanks!

Problems installing on Ubuntu 22.04

Tried to install as described, I get this:

michael@michael-kontor:~/github/stable_diffusion.openvino$ pip install -r requirements.txt 
Defaulting to user installation because normal site-packages is not writeable
Collecting numpy==1.19.5
  Using cached numpy-1.19.5.zip (7.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting opencv-python==4.5.5.64
  Using cached opencv_python-4.5.5.64-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.5 MB)
Collecting transformers==4.16.2
  Using cached transformers-4.16.2-py3-none-any.whl (3.5 MB)
Collecting diffusers==0.2.4
  Using cached diffusers-0.2.4-py3-none-any.whl (112 kB)
Collecting tqdm==4.64.0
  Using cached tqdm-4.64.0-py2.py3-none-any.whl (78 kB)
ERROR: Could not find a version that satisfies the requirement openvino==2022.1.0 (from versions: none)
ERROR: No matching distribution found for openvino==2022.1.0

Info about my system:

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.1 LTS
Release:	22.04
Codename:	jammy

$ uname -a
Linux michael-kontor 5.15.0-47-generic #51-Ubuntu SMP Thu Aug 11 07:51:15 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

$ pip --version
pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)

$ cat /proc/cpuinfo
(some info left out)
processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 158
model name	: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
stepping	: 9
microcode	: 0xf0
cpu MHz		: 4200.000
cache size	: 8192 KB

Not much help to be found online regarding openvino except some lack of support with CentOS which I don't think is relevant on my machine.

Any ideas?

More ram more speed?

is it possible to increase ram consumption to speed up inference?

Selecting different samplers?

Some of SD interfaces expose the ability to select the sampler. In particular, the k_euler_a is known to produce good results with smaller number of iterations, which would be fantastic for CPU version.

Where does this framework select the sampler implementation? Could it be exposed in the shell arguments?

Add seed option to streamlit GUI

It would be great if the GUI had the option to specify the seed as well for consistent prompt generation, considering it already has all the options exposed by demo.py (strength, steps, etc.)

fish: Job 1, 'python3 stable_diffusion.py --p…' terminated by signal SIGKILL (Forced quit)

I get a persistent error when my job finishes... the process is killed, and no image is outputted.

Here's an example using 2 inference steps only (so it runs quickly):

> python3 stable_diffusion.py --prompt "Street-art painting of a dinosaur in the style of Banksy, photorealism" --output "banksy-dinosaur.jpg" --num-inference-steps 2
2it [00:25, 12.97s/it]
fish: Job 1, 'python3 stable_diffusion.py --p…' terminated by signal SIGKILL (Forced quit)

The same thing happens in both fish and bash.

What is causing this, and how can I get the job to complete?

Document parameter defaults

Especially these two seem interesting

  --num-inference-steps NUM_INFERENCE_STEPS
                        num inference steps
  --guidance-scale GUIDANCE_SCALE
                        guidance scale

By limiting the inference steps, you seem to get an intermediate result faster.
Is there any other way to get a quick prototype, maybe in lower resolution or something?

Does this project work on Mac M1 CPU?

I tried but always failed in below line of code:
self.unet = self.core.compile_model(self._unet, device)
There is error when compiling unet, with error "node input index is out of range".
Does the error due to unsupported M1 CPU?
Thanks.

img to img support

Stable diffusion has img to img, is that possible with this version?

25% utilitization - 2/8 cores

I have an Intel Xeon with 16 cores. It's running proxmox, 8 cores exposed to a container in a LXC (confirmed with /proc/cpuinfo).

This project/OpenVino is running on just 2/8 cores and top reports 25% utilization.

Is there any way to tell it to use all the cores available?

Can I open a PR for poetry?

Hi @bes-dev,

Thank you for your great work.
I tried your repo with poetry.
Do you think if I open a PR for poetry, it will be useful for others?

Suggestion: add argument for number of images

If you want to see the same prompt run multiple times with different seed, you have to do it manually right now and remember to change the output file name. With an argument for number of images, it would automatically re-run the same prompt that number of times, automatically adding "_1" etc to the output filename so they don't get overwritten.

Documenting installation issues

I'm just documenting some issues I ran into while installing, and what the fixes were!

Openvino version cannot be found.

Problem:

ERROR: Could not find a version that satisfies the requirement openvino==2022.1.0 (from -r requirements.txt (line 6)) (from versions: 2021.3.0, 2021.4.0, 2021.4.1, 2021.4.2)
ERROR: No matching distribution found for openvino==2022.1.0 (from -r requirements.txt (line 6))

Solution: Upgrade pip.

> pip install --upgrade pip
> pip install -r requirements.txt

Character encoding issues.

Problem:

> python stable_diffusion.py --prompt "Street-art painting of Emilia Clarke in style of Banksy, photorealism"
  File "stable_diffusion.py", line 92
SyntaxError: Non-ASCII character '\xce' in file stable_diffusion.py on line 92, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Solution: Add the following to the top of stable_diffusion.py: # -*- coding: utf-8 -*-

> nano stable_diffusion.py

No module named `numpy`.

Problem:

> python stable_diffusion.py --prompt "Street-art painting of Emilia Clarke in st
yle of Banksy, photorealism"
Traceback (most recent call last):
  File "stable_diffusion.py", line 6, in <module>
    import numpy as np
ImportError: No module named numpy

Solution: Ensure you are using python3.

> python -V
Python 2.7.18
> python3 stable_diffusion.py --prompt "Street-art painting of Emilia Clarke in style of Banksy, photorealism"

Suggestion: Add --height and --width for generating images in different pixel sizes

# Height Width parser.add_argument("--H", type=int, default=1280, help="image height, in pixel space") parser.add_argument("--W", type=int, default=720, help="image width, in pixel space")

Some prompts and number of iterations result in an entirely black image

For some prompts like "Superman in the mountains" (32 iterations) I get a black image as result. Is that to be expected? The same on an Intel and AMD CPUs.
For other prompts or fewer --num-inference-steps (e.g. 4 which is < 32), it works but with an uglier image.
Any clues?

Can it be made to run entirely offline?

Hello, thanks for the very useful project! :)
Right now, after each run on the console, it takes ~30 seconds contacting some third-party websites before the actual inference starts.
I am on a slow, laggy Internet connection and it's really annoying to wait each time. If I disconnect from the Internet entirely, then the tool complains about "cannot find the requested files in the cached path" but they must be there (in the cache) already since it no longer downloads anything...

Also, for some prompts like "Superman in the mountains" I get a black image as result. Is that to be expected?

Suggestion: Add img2img and inpaint scripts

Inpainting (--mask) does not work

I’ve tried the command line specified in the README.md, and I’ve tried some images+masks of my own. I only seem to get variations of the original image, meaning that the mask is being ignored.

I’ve looked at the _preprocess_mask function and had it output its result. I couldn’t find anything wrong with it. Despite, the output does not appear to use it.

Thanks.

Your app is having trouble loading the streamlit_drawable_canvas.st_canvas component

I get this message in the web ui when I click on init_image.

Your app is having trouble loading the streamlit_drawable_canvas.st_canvas component.

(The app is attempting to load the component from ****, and hasn't received its "streamlit:componentReady" message.)

If this is a development build, have you started the dev server?
If this is a release build, have you compiled the frontend?
For more troubleshooting help, please see the [Streamlit Component docs](https://docs.streamlit.io/library/components) or visit our [forums](https://discuss.streamlit.io/).

New generations start by itself, without my initiative after I pressed the button.
When the generation is completed, and the next one immediately begins, and so on indefinitely, until I interrupt the execution of the script in console. output.png is not overwritten, new images do not appear in the browser window.
When I change the generation settings in the UI, they affect the next generation (in any case, the number of steps), but incorrectly. For example, I changed the number of steps from 32 to 31, and the generation beganwith 18 steps.

After a few iterations, the gui for inpainting loaded, and I drew a mask on the image that I downloaded earlier (which probably started an endless cycle of generations), and the new generations that I see in the browser console match the expectation from such inpainting.

Gather usage stats:  true
5.0803492a.chunk.js:2 Unrecognized feature: 'ambient-light-sensor'.
A @ 5.0803492a.chunk.js:2
5.0803492a.chunk.js:2 Unrecognized feature: 'battery'.
A @ 5.0803492a.chunk.js:2
5.0803492a.chunk.js:2 Unrecognized feature: 'layout-animations'.
A @ 5.0803492a.chunk.js:2
5.0803492a.chunk.js:2 Unrecognized feature: 'legacy-image-formats'.
A @ 5.0803492a.chunk.js:2
5.0803492a.chunk.js:2 Unrecognized feature: 'oversized-images'.
A @ 5.0803492a.chunk.js:2
5.0803492a.chunk.js:2 Unrecognized feature: 'vr'.
A @ 5.0803492a.chunk.js:2
5.0803492a.chunk.js:2 Unrecognized feature: 'wake-lock'.
A @ 5.0803492a.chunk.js:2
DevTools failed to load source map: Could not load content for http://localhost:8501/component/streamlit_drawable_canvas.st_canvas/static/js/2.5ffbf28f.chunk.js.map: Load canceled due to load timeout
DevTools failed to load source map: Could not load content for http://localhost:8501/component/streamlit_drawable_canvas.st_canvas/static/js/main.fac01c65.chunk.js.map: Load canceled due to load timeout
main.fac01c65.chunk.js:1 /media/95f25f493988202165a0299ea05735fabf533c46b092f4013a64b030.png
main.fac01c65.chunk.js:1 {host: 'localhost', port: 8501, basePath: 'component/streamlit_drawable_canvas.st_canvas/index.html'}
main.fac01c65.chunk.js:1 http://localhost:8501/media/95f25f493988202165a0299ea05735fabf533c46b092f4013a64b030.png

cython dependency not in requirements.txt

Suggestion: add Textual Inversion

ONNX models

First of all, thank you for this, tried it out on CPU and there was some speed boost compared to pytroch.

However i am wondering if you would be willing to share the possible onnx-versions of the models?
I don't have a lot of experience with openvino, but to my understanding converting to onnx is usually a step in converting to the openvino format?
Having the onnx ones may also, with some luck help amd users to run it eventually.

ModuleNotFoundError: No module named 'openvino.runtime'

Any one know this below?

Ubuntu 20.04 LTS

root@VM-8-7-ubuntu:~/stable_diffusion.openvino# python demo.py --prompt "Street-art painting of Emilia Clarke in style of Banksy, photorealism"
Traceback (most recent call last):
File "demo.py", line 5, in
from stable_diffusion_engine import StableDiffusionEngine
File "/root/stable_diffusion.openvino/stable_diffusion_engine.py", line 4, in
from openvino.runtime import Core
ModuleNotFoundError: No module named 'openvino.runtime'

root@VM-8-7-ubuntu:/stable_diffusion.openvino# pip install openvino
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already satisfied: openvino in /usr/local/lib/python3.8/dist-packages (2021.4.2)
Requirement already satisfied: numpy<1.20,>=1.16.6 in /usr/local/lib/python3.8/dist-packages (from openvino) (1.19.5)
root@VM-8-7-ubuntu:/stable_diffusion.openvino#

Has this model been optimized in OpenVINO?

Looking over the OpenVino documentation there seems to be a lot of potential for optimization, especially optimization of an already trained model:

https://docs.openvino.ai/latest/openvino_docs_model_optimization_guide.html

My initial question here would be: is the model used here the original SD one? Or has it been run through any of the OpenVino optimization tools?

Parameter for multiple image generation like --images 100

Please, if possible i will ask to you please develop a function that make possible to make multiple images. Thanks.

getting the following error

huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error: Entry Not Found for url: https://huggingface.co/bes-dev/stable-diffusion-v1-4-openvino/resolve/main/vae.xml. (Request ID: meIMbxjucCK0UlHWmBiba)

use gradio for UI

Hi, there are some nice projects like https://github.com/hlky/stable-diffusion-webui and https://github.com/altryne/sd-webui-colab, which support a gradio web UI, gradio has many features like blocks which allows multiple apps in one and queueing, caching examples, concurrent requests etc

Could not find a version that satisfies the requirement openvino==2022.1.0

ERROR: Could not find a version that satisfies the requirement openvino==2022.1.0 (from -r requirements.txt (line 6)) (from versions: 2021.2, 2021.3.0, 2021.4.0, 2021.4.1, 2021.4.2)
ERROR: No matching distribution found for openvino==2022.1.0 (from -r requirements.txt (line 6))

stroke_color can't be modified in web app

The web provides the option to change stroke_color, but the color always remains green

ModuleNotFoundError: No module named 'streamlit_drawable_canvas'

There is no streamlit_drawable_canvas library in requirements.txt

2022-09-03 08:36:23.931 Uncaught app exception
Traceback (most recent call last):
  File "C:\Users\User\stable_diffusion.openvino-master\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 556, in _run_script
    exec(code, module.__dict__)
  File "C:\Users\User\stable_diffusion.openvino-master\demo_web.py", line 6, in <module>
    from streamlit_drawable_canvas import st_canvas
ModuleNotFoundError: No module named 'streamlit_drawable_canvas'
  Stopping...

Installing streamlit_drawable_canvas solves this problem.

Uncensor

the project is very good but it look has it have some censorship about nudity. Are possible to turn off al the filters from the models used?

Auto rename image

Just a suggestion, if possible, if an image output already exist, it will be renamed to output1, output2. Thanks again.

Using Doggettx's Automatic memory management fork?

https://github.com/Doggettx/stable-diffusion
Automatic memory management
This version can be run stand alone, but it's more meant as proof of concept so other forks can implement similar changes.
Allows to use resolutions that require up to 64x more VRAM than possible on the default CompVis build.
As Openvino using CPU, maybe it can use the full system RAM for generating?
Thanks!

bes-dev / stable_diffusion.openvino Goto Github PK

stable_diffusion.openvino's Introduction

stable_diffusion.openvino

Requirements

Install requirements

Generate image from text description

Examples

Example Text-To-Image

Example Image-To-Image

Example Inpainting

Performance

Acknowledgements

Disclaimer

stable_diffusion.openvino's People

Contributors

Stargazers

Watchers

Forkers

stable_diffusion.openvino's Issues

Openvino version cannot be found.

Character encoding issues.

No module named numpy.

Recommend Projects

Recommend Topics

Recommend Org

No module named `numpy`.