multimodalart / majesty-diffusion Goto Github PK

View Code? Open in Web Editor NEW

275.0 275.0 45.0 183 KB

Majesty Diffusion by @Dango233(@Dango233max) and @apolinario (@multimodalart)

Jupyter Notebook 100.00%

majesty-diffusion's People

Contributors

Stargazers

Watchers

majesty-diffusion's Issues

does init image have a big effect? can it be a url link rather than image path?

i mean this one

AI do not understand the word "holding" in prompt "holding a sword"

prompt: A warrior wombat holding a sword and shield in a fighting stance. The wombat stands in front of the Arc de Triomphe on a day shrouded mist with the sun high in the sky. realistic anime illustration.

it should produce image like this

Deleting Mask File Path or entering "None" causes error

Easily fixed by copying the same code above it for the init_image:

if(init_mask == '' or init_mask == 'None'): init_mask = None

incompatible package versions

Hi, I'm working on a project and want a specific look that I was only able to generate with majesty confusion however it seems that six months later, the Colab doesn't work because not all of the packages were pinned to specific versions.

Could someone provide me a pip list from a working repo so I can reconfigure the Colab to work again?

Colab error : Basicsr module not found and then crashes

I was playing since some days with my own instance on Colab, all was working like a charm, but today, it doesn't work anymore.
Fearing I have misstyped something, I went back to the original instance hosted here, but I'm still getting the same error, that is happening when using GFPGAN upscaling:

Sampling images 1/1
Data shape for DDIM sampling is (1, 4, 32, 32), eta 1.3
Running DDIM Sampling with 119 timesteps
DDIM Sampler: 100%
119/119 [02:06<00:00, 1.01it/s]
/content/GFPGAN
Traceback (most recent call last):
  File "inference_gfpgan.py", line 9, in <module>
    from gfpgan import GFPGANer
  File "/content/GFPGAN/gfpgan/__init__.py", line 4, in <module>
    from .models import *
  File "/content/GFPGAN/gfpgan/models/__init__.py", line 10, in <module>
    _model_modules = [importlib.import_module(f'gfpgan.models.{file_name}') for file_name in model_filenames]
  File "/content/GFPGAN/gfpgan/models/__init__.py", line 10, in <listcomp>
    _model_modules = [importlib.import_module(f'gfpgan.models.{file_name}') for file_name in model_filenames]
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/content/GFPGAN/gfpgan/models/gfpgan_model.py", line 6, in <module>
    from basicsr.losses.losses import r1_penalty
ModuleNotFoundError: No module named 'basicsr.losses.losses'
/content
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-13-6ec39f84cd27>](https://localhost:8080/#) in <module>()
    147       fps = 24
    148       p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'png', '-r', str(fps), '-i', '-', '-vcodec', 'libx264', '-r', str(fps), '-pix_fmt', 'yuv420p', '-crf', '17', '-preset', 'veryslow', 'video.mp4'], stdin=PIPE)
--> 149   do_run()
    150   if generate_video:
    151       p.stdin.close()

1 frames
[<ipython-input-8-0502eee025d5>](https://localhost:8080/#) in fetch(url_or_path)
    124         fd.seek(0)
    125         return fd
--> 126     return open(url_or_path, 'rb')
    127 
    128 

FileNotFoundError: [Errno 2] No such file or directory: 'GFPGAN/results/restored_imgs/temp_1657646799.png'

Replace the Multi-Modal-Comparators module with a simple script

Hi I'm trying to run the majesty-diffusion locally, but I have some challenges on the Multi-Modal-Comparators.

The Multi-Modal-Comparators have some issues:

It has some version conflicts. The mmc depend on Pillow-7.2.0, but the minimum version for python 3.10 is Pillow 8.3.2 https://pillow.readthedocs.io/en/stable/installation.html
It's too complicated. Clip models management could be very straightforward. Progrockdiffusion project has a 200 lines scripts to manage models and it works very well. https://github.com/lowfuel/progrockdiffusion/blob/main/model_managers/clip_manager.py

Thanks.

MinDalle support?

I was wondering if there is a way to add the MinDalle model to the list of droptdowns for additional models.

Model memorizes previous run

Reported by @aiartdue77.

Seeding the system

Hi, great work on this system!

I am looking for a way to make the image generation reproducible.
Other notebooks/scripts have a seed value. I added some code to Latent Majesty Diffusion (v1.3) to seed torch:

seed = 1234567
torch.manual_seed(seed)

I also added this snippet to seed pseudo-random number generators in pytorch, numpy, python.random:

from pytorch_lightning import seed_everything
seed_everything(seed, workers=True)

But the images are still different in each run.

Any idea what else I need to change to make the image generation reproducible? That is, given the same seed, I want the system to produce the same image.

Memory leak

Running with how_many_batches>1 can lead to "Cuda out of memory" errors after the first image is completed. There is a memory leak somewhere.

Other notebooks solve this problem by putting the batch "for" loop around the do_run() function rather than inside the do_run() function. That way, whatever mystery local variable that causes this problem gets cleaned up.

TypeError: modify_score() argument after ** must be a mapping, not list

btw, opt.ddim_steps is not specified

Got this TypeError for V-Majesty Notebook

I tried to do the run with all default settings and when I wanted to get into the diffusion, I got this:

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

[<ipython-input-25-4d917cc6e118>](https://localhost:8080/#) in <module>()
    141                     gc.collect()
    142                     torch.cuda.empty_cache()
--> 143                     do_run()
    144                     #threading.Thread(target=do_run, args=()).start()
    145 

5 frames

[<ipython-input-16-ced52f73bed5>](https://localhost:8080/#) in forward(self, input)
     75         sideY, sideX = input.shape[2:4]
     76         max_size = min(sideX, sideY)
---> 77         min_size = min(sideX, sideY, self.cut_size)
     78         l_size = max(sideX, sideY)
     79         output_shape = [1,3,self.cut_size,self.cut_size]

TypeError: '<' not supported between instances of 'tuple' and 'int'

If there is any fix to this, I'd be happy to be noticed.

Install fails with V100

This issue hapens on the unmodified Colab when I'm pressing "run all":https://colab.research.google.com/github/multimodalart/MajestyDiffusion/blob/main/latent.ipynb

It seems to only happen with an V100? But I have not explored all the cases…
The error appears in the Installation cell

I am uncertain if this part is relevant:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
xarray-einstats 0.2.2 requires numpy>=1.21, but you have numpy 1.20.3 which is incompatible.
torchtext 0.13.0 requires torch==1.12.0, but you have torch 1.10.2 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
Successfully installed gfpgan-1.3.2 numpy-1.20.3 realesrgan-0.2.5.0
WARNING: The following packages were previously imported in this runtime:
  [numpy]
You must restart the runtime in order to use newly installed versions.
---------------------------------------------------------------------------

But this should not happen:

OSError                                   Traceback (most recent call last)
<ipython-input-5-d9296e7aa8b5> in <module>()
     25 
     26     sys.path.append('./taming-transformers')
---> 27     from taming.models import vqgan
     28     from subprocess import Popen, PIPE
     29     try:

14 frames
/usr/lib/python3.7/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error)
    362 
    363         if handle is None:
--> 364             self._handle = _dlopen(self._name, mode)
    365         else:
    366             self._handle = handle

OSError: /usr/local/lib/python3.7/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN3c1022getCustomClassTypeImplERKSt10type_index

Full output of the Installation cell: Click to expand!

Cloning into 'latent-diffusion'...
remote: Enumerating objects: 346, done.
remote: Counting objects: 100% (51/51), done.
remote: Compressing objects: 100% (35/35), done.
remote: Total 346 (delta 22), reused 37 (delta 16), pack-reused 295
Receiving objects: 100% (346/346), 28.43 MiB | 25.38 MiB/s, done.
Resolving deltas: 100% (135/135), done.
Cloning into 'taming-transformers'...
remote: Enumerating objects: 1335, done.
remote: Total 1335 (delta 0), reused 0 (delta 0), pack-reused 1335
Receiving objects: 100% (1335/1335), 409.77 MiB | 33.39 MiB/s, done.
Resolving deltas: 100% (278/278), done.
Cloning into 'GFPGAN'...
remote: Enumerating objects: 431, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 431 (delta 0), reused 4 (delta 0), pack-reused 426
Receiving objects: 100% (431/431), 5.36 MiB | 29.82 MiB/s, done.
Resolving deltas: 100% (214/214), done.
WARNING: 'git lfs clone' is deprecated and will not be updated
         with new flags from 'git clone'

'git clone' has been updated in upstream Git to have comparable
speeds to 'git lfs clone'.
Cloning into 'latent-majesty-diffusion-settings'...
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 14 (delta 4), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (14/14), done.
WARNING: 'git lfs clone' is deprecated and will not be updated
         with new flags from 'git clone'

'git clone' has been updated in upstream Git to have comparable
speeds to 'git lfs clone'.
Cloning into 'aesthetic-predictor'...
remote: Enumerating objects: 67, done.
remote: Counting objects: 100% (67/67), done.
remote: Compressing objects: 100% (64/64), done.
remote: Total 67 (delta 21), reused 28 (delta 3), pack-reused 0
Unpacking objects: 100% (67/67), done.
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Obtaining file:///content/taming-transformers
Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from taming-transformers==0.0.1) (1.10.2)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from taming-transformers==0.0.1) (1.21.6)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from taming-transformers==0.0.1) (4.64.0)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch->taming-transformers==0.0.1) (4.1.1)
Installing collected packages: taming-transformers
 Running setup.py develop for taming-transformers
Successfully installed taming-transformers-0.0.1
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting transformers
 Downloading transformers-4.20.1-py3-none-any.whl (4.4 MB)
    |████████████████████████████████| 4.4 MB 14.7 MB/s 
Collecting huggingface-hub<1.0,>=0.1.0
 Downloading huggingface_hub-0.8.1-py3-none-any.whl (101 kB)
    |████████████████████████████████| 101 kB 13.1 MB/s 
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (21.3)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers) (4.12.0)
Collecting tokenizers!=0.11.3,<0.13,>=0.11.1
 Downloading tokenizers-0.12.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB)
    |████████████████████████████████| 6.6 MB 60.1 MB/s 
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.21.6)
Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.7.1)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2022.6.2)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (6.0)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.64.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (4.1.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->transformers) (3.0.9)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers) (3.8.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.24.3)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2022.6.15)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10)
Installing collected packages: tokenizers, huggingface-hub, transformers
Successfully installed huggingface-hub-0.8.1 tokenizers-0.12.1 transformers-4.20.1
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting dotmap
 Downloading dotmap-1.3.30-py3-none-any.whl (11 kB)
Installing collected packages: dotmap
Successfully installed dotmap-1.3.30
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting resize-right
 Downloading resize_right-0.0.2-py3-none-any.whl (8.9 kB)
Installing collected packages: resize-right
Successfully installed resize-right-0.0.2
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting piq
 Downloading piq-0.7.0-py3-none-any.whl (139 kB)
    |████████████████████████████████| 139 kB 14.0 MB/s 
Requirement already satisfied: torchvision!=0.9.*,>=0.6.1 in /usr/local/lib/python3.7/dist-packages (from piq) (0.11.3)
Requirement already satisfied: torch==1.10.2 in /usr/local/lib/python3.7/dist-packages (from torchvision!=0.9.*,>=0.6.1->piq) (1.10.2)
Requirement already satisfied: pillow!=8.3.0,>=5.3.0 in /usr/local/lib/python3.7/dist-packages (from torchvision!=0.9.*,>=0.6.1->piq) (7.1.2)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torchvision!=0.9.*,>=0.6.1->piq) (1.21.6)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch==1.10.2->torchvision!=0.9.*,>=0.6.1->piq) (4.1.1)
Installing collected packages: piq
Successfully installed piq-0.7.0
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting lpips
 Downloading lpips-0.1.4-py3-none-any.whl (53 kB)
    |████████████████████████████████| 53 kB 2.5 MB/s 
Requirement already satisfied: numpy>=1.14.3 in /usr/local/lib/python3.7/dist-packages (from lpips) (1.21.6)
Requirement already satisfied: scipy>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from lpips) (1.5.4)
Requirement already satisfied: torchvision>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from lpips) (0.11.3)
Requirement already satisfied: tqdm>=4.28.1 in /usr/local/lib/python3.7/dist-packages (from lpips) (4.64.0)
Requirement already satisfied: torch>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from lpips) (1.10.2)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch>=0.4.0->lpips) (4.1.1)
Requirement already satisfied: pillow!=8.3.0,>=5.3.0 in /usr/local/lib/python3.7/dist-packages (from torchvision>=0.2.1->lpips) (7.1.2)
Installing collected packages: lpips
Successfully installed lpips-0.1.4
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting basicsr
 Downloading basicsr-1.4.0.tar.gz (171 kB)
    |████████████████████████████████| 171 kB 14.0 MB/s 
Collecting addict
 Downloading addict-2.4.0-py3-none-any.whl (3.8 kB)
Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from basicsr) (0.16.0)
Requirement already satisfied: lmdb in /usr/local/lib/python3.7/dist-packages (from basicsr) (0.99)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from basicsr) (1.21.6)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (from basicsr) (4.1.2.30)
Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from basicsr) (7.1.2)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from basicsr) (6.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from basicsr) (2.23.0)
Requirement already satisfied: scikit-image in /usr/local/lib/python3.7/dist-packages (from basicsr) (0.18.3)
Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from basicsr) (1.5.4)
Collecting tb-nightly
 Downloading tb_nightly-2.10.0a20220712-py3-none-any.whl (5.8 MB)
    |████████████████████████████████| 5.8 MB 89.8 MB/s 
Requirement already satisfied: torch>=1.7 in /usr/local/lib/python3.7/dist-packages (from basicsr) (1.10.2)
Requirement already satisfied: torchvision in /usr/local/lib/python3.7/dist-packages (from basicsr) (0.11.3)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from basicsr) (4.64.0)
Collecting yapf
 Downloading yapf-0.32.0-py2.py3-none-any.whl (190 kB)
    |████████████████████████████████| 190 kB 91.5 MB/s 
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch>=1.7->basicsr) (4.1.1)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->basicsr) (1.24.3)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->basicsr) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->basicsr) (2022.6.15)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->basicsr) (3.0.4)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr) (1.3.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr) (2021.11.2)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr) (2.6.3)
Requirement already satisfied: matplotlib!=3.0.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr) (3.2.2)
Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr) (2.4.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->basicsr) (0.11.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->basicsr) (2.8.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->basicsr) (3.0.9)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->basicsr) (1.4.3)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib!=3.0.0,>=2.0.0->scikit-image->basicsr) (1.15.0)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (1.1.0)
Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (1.47.0)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (57.4.0)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (1.0.1)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (1.35.0)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (1.8.1)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (0.37.1)
Requirement already satisfied: protobuf<3.20,>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (3.17.3)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (0.6.1)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (3.3.7)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr) (0.4.6)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tb-nightly->basicsr) (4.2.4)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tb-nightly->basicsr) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tb-nightly->basicsr) (4.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tb-nightly->basicsr) (1.3.1)
Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tb-nightly->basicsr) (4.12.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tb-nightly->basicsr) (3.8.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tb-nightly->basicsr) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tb-nightly->basicsr) (3.2.0)
Building wheels for collected packages: basicsr
 Building wheel for basicsr (setup.py) ... done
 Created wheel for basicsr: filename=basicsr-1.4.0-py3-none-any.whl size=213769 sha256=cec57b47cf1c18b06ff5f0b8ea6e01a7995a61320330851d539f09fc01a90352
 Stored in directory: /root/.cache/pip/wheels/ab/0f/9e/e62b449971d38bbefbb54fe838167b91341f7e18b34270525f
Successfully built basicsr
Installing collected packages: yapf, tb-nightly, addict, basicsr
Successfully installed addict-2.4.0 basicsr-1.4.0 tb-nightly-2.10.0a20220712 yapf-0.32.0
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting facexlib
 Downloading facexlib-0.2.4-py3-none-any.whl (59 kB)
    |████████████████████████████████| 59 kB 5.7 MB/s 
Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (from facexlib) (1.10.2)
Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from facexlib) (1.5.4)
Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from facexlib) (7.1.2)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from facexlib) (4.64.0)
Requirement already satisfied: torchvision in /usr/local/lib/python3.7/dist-packages (from facexlib) (0.11.3)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (from facexlib) (4.1.2.30)
Collecting filterpy
 Downloading filterpy-1.4.5.zip (177 kB)
    |████████████████████████████████| 177 kB 29.0 MB/s 
Requirement already satisfied: numba in /usr/local/lib/python3.7/dist-packages (from facexlib) (0.51.2)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from facexlib) (1.21.6)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from filterpy->facexlib) (3.2.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->filterpy->facexlib) (0.11.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->filterpy->facexlib) (2.8.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->filterpy->facexlib) (3.0.9)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->filterpy->facexlib) (1.4.3)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib->filterpy->facexlib) (4.1.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->filterpy->facexlib) (1.15.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from numba->facexlib) (57.4.0)
Requirement already satisfied: llvmlite<0.35,>=0.34.0.dev0 in /usr/local/lib/python3.7/dist-packages (from numba->facexlib) (0.34.0)
Building wheels for collected packages: filterpy
 Building wheel for filterpy (setup.py) ... done
 Created wheel for filterpy: filename=filterpy-1.4.5-py3-none-any.whl size=110474 sha256=208ebc749b8f47447004807d1cd1f609c84f3ccabb646ef0fae220405e4eb7a0
 Stored in directory: /root/.cache/pip/wheels/ce/e0/ee/a2b3c5caab3418c1ccd8c4de573d4cbe13315d7e8b0a55fbc2
Successfully built filterpy
Installing collected packages: filterpy, facexlib
Successfully installed facexlib-0.2.4 filterpy-1.4.5
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting realesrgan
 Downloading realesrgan-0.2.5.0-py3-none-any.whl (25 kB)
Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from realesrgan) (7.1.2)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/dist-packages (from realesrgan) (4.1.2.30)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from realesrgan) (1.21.6)
Requirement already satisfied: torch>=1.7 in /usr/local/lib/python3.7/dist-packages (from realesrgan) (1.10.2)
Requirement already satisfied: facexlib>=0.2.0.3 in /usr/local/lib/python3.7/dist-packages (from realesrgan) (0.2.4)
Requirement already satisfied: basicsr>=1.3.3.11 in /usr/local/lib/python3.7/dist-packages (from realesrgan) (1.4.0)
Requirement already satisfied: torchvision in /usr/local/lib/python3.7/dist-packages (from realesrgan) (0.11.3)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from realesrgan) (4.64.0)
Collecting gfpgan>=0.2.1
 Downloading gfpgan-1.3.2-py3-none-any.whl (47 kB)
    |████████████████████████████████| 47 kB 5.8 MB/s 
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (6.0)
Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (0.16.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (2.23.0)
Requirement already satisfied: yapf in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (0.32.0)
Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (1.5.4)
Requirement already satisfied: addict in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (2.4.0)
Requirement already satisfied: scikit-image in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (0.18.3)
Requirement already satisfied: lmdb in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (0.99)
Requirement already satisfied: tb-nightly in /usr/local/lib/python3.7/dist-packages (from basicsr>=1.3.3.11->realesrgan) (2.10.0a20220712)
Requirement already satisfied: filterpy in /usr/local/lib/python3.7/dist-packages (from facexlib>=0.2.0.3->realesrgan) (1.4.5)
Requirement already satisfied: numba in /usr/local/lib/python3.7/dist-packages (from facexlib>=0.2.0.3->realesrgan) (0.51.2)
Collecting numpy
 Downloading numpy-1.20.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.3 MB)
    |████████████████████████████████| 15.3 MB 29.0 MB/s 
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch>=1.7->realesrgan) (4.1.1)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from filterpy->facexlib>=0.2.0.3->realesrgan) (3.2.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->filterpy->facexlib>=0.2.0.3->realesrgan) (3.0.9)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->filterpy->facexlib>=0.2.0.3->realesrgan) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->filterpy->facexlib>=0.2.0.3->realesrgan) (1.4.3)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->filterpy->facexlib>=0.2.0.3->realesrgan) (0.11.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->filterpy->facexlib>=0.2.0.3->realesrgan) (1.15.0)
Requirement already satisfied: llvmlite<0.35,>=0.34.0.dev0 in /usr/local/lib/python3.7/dist-packages (from numba->facexlib>=0.2.0.3->realesrgan) (0.34.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from numba->facexlib>=0.2.0.3->realesrgan) (57.4.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->basicsr>=1.3.3.11->realesrgan) (2.10)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->basicsr>=1.3.3.11->realesrgan) (1.24.3)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->basicsr>=1.3.3.11->realesrgan) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->basicsr>=1.3.3.11->realesrgan) (2022.6.15)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr>=1.3.3.11->realesrgan) (2.6.3)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr>=1.3.3.11->realesrgan) (1.3.0)
Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr>=1.3.3.11->realesrgan) (2.4.1)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/dist-packages (from scikit-image->basicsr>=1.3.3.11->realesrgan) (2021.11.2)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (1.8.1)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (3.3.7)
Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (1.47.0)
Requirement already satisfied: protobuf<3.20,>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (3.17.3)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (0.4.6)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (0.37.1)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (1.1.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (1.35.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (0.6.1)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from tb-nightly->basicsr>=1.3.3.11->realesrgan) (1.0.1)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tb-nightly->basicsr>=1.3.3.11->realesrgan) (4.2.4)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tb-nightly->basicsr>=1.3.3.11->realesrgan) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tb-nightly->basicsr>=1.3.3.11->realesrgan) (0.2.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tb-nightly->basicsr>=1.3.3.11->realesrgan) (1.3.1)
Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tb-nightly->basicsr>=1.3.3.11->realesrgan) (4.12.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tb-nightly->basicsr>=1.3.3.11->realesrgan) (3.8.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tb-nightly->basicsr>=1.3.3.11->realesrgan) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tb-nightly->basicsr>=1.3.3.11->realesrgan) (3.2.0)
Installing collected packages: numpy, gfpgan, realesrgan
 Attempting uninstall: numpy
   Found existing installation: numpy 1.21.6
   Uninstalling numpy-1.21.6:
     Successfully uninstalled numpy-1.21.6
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
xarray-einstats 0.2.2 requires numpy>=1.21, but you have numpy 1.20.3 which is incompatible.
torchtext 0.13.0 requires torch==1.12.0, but you have torch 1.10.2 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
Successfully installed gfpgan-1.3.2 numpy-1.20.3 realesrgan-0.2.5.0
WARNING: The following packages were previously imported in this runtime:
 [numpy]
You must restart the runtime in order to use newly installed versions.
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
[<ipython-input-4-d9296e7aa8b5>](https://localhost:8080/#) in <module>()
    25 
    26     sys.path.append('./taming-transformers')
---> 27     from taming.models import vqgan
    28     from subprocess import Popen, PIPE
    29     try:

14 frames
[/usr/lib/python3.7/ctypes/__init__.py](https://localhost:8080/#) in __init__(self, name, mode, handle, use_errno, use_last_error)
   362 
   363         if handle is None:
--> 364             self._handle = _dlopen(self._name, mode)
   365         else:
   366             self._handle = handle

OSError: /usr/local/lib/python3.7/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN3c1022getCustomClassTypeImplERKSt10type_index

Make the model(s) available for diffusers.

I recently found about huggingface/diffusers and it is one of the best ways for inference of a diffusion model in my opinion. If it's possible, please make the model available for diffusers as well.

EOF error at the Load Necessary Functions step

Getting an EOF error at the Load Necessary Functions step, Line 461.

Stack exchange says it's probably a pickle being overwritten but that's as far as my trouble shooting skill can take me.

Tried it in two separate colab instances, one with the Erlich model, the second with finetuned.

Thank you :)

`Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
Loading model from: /usr/local/lib/python3.7/dist-packages/lpips/weights/v0.1/vgg.pth
---------------------------------------------------------------------------
EOFError                                  Traceback (most recent call last)
[<ipython-input-14-f555e4f938e4>](https://localhost:8080/#) in <module>()
    459 #Alstro's aesthetic model
    460 aesthetic_model_336 = torch.nn.Linear(768,1).cuda()
--> 461 aesthetic_model_336.load_state_dict(torch.load(f"{model_path}/ava_vit_l_14_336_linear.pth"))
    462 
    463 aesthetic_model_224 = torch.nn.Linear(768,1).cuda()

1 frames
[/usr/local/lib/python3.7/dist-packages/torch/serialization.py](https://localhost:8080/#) in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
    775             "functionality.")
    776 
--> 777     magic_number = pickle_module.load(f, **pickle_load_args)
    778     if magic_number != MAGIC_NUMBER:
    779         raise RuntimeError("Invalid magic number; corrupt file?")

EOFError: Ran out of input`

last output is always blank

Hi,
I tried to run the notebook with sample prompts but the last saved output is always blank. Also the quality of generated images is not even better than latent diffusion with jack model. Do you think its a problem with settings or something else?
Thanks

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.