CUDA runs out of memory with lots of memory reserved about stable-diffusion HOT 11 OPEN

Having the same problem
RuntimeError: CUDA out of memory. Tried to allocate 114.00 MiB (GPU 0; 8.00 GiB total capacity; 7.14 GiB already allocated; 0 bytes free; 7.24 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

from stable-diffusion.

The same model (latent diffusion 1.6B) does run on 8 GB when using Jack000/glid-3-xl, so it is supposed to work.

from stable-diffusion.

I'm having the same problem it allocates tons of memory and then fails

from stable-diffusion.

Also runs out of VRAM on a 16 GB P100, so something is definitely wrong. This did not happen with the same model on the latent-diffusion repo
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 15.90 GiB total capacity; 15.07 GiB already allocated; 21.75 MiB free; 15.25 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

from stable-diffusion.

It appears I did have the problem on the latent-diffusion repo as well, but I fixed my problem on both by adding os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "max_split_size_mb:4096" near the start of text2img.py

from stable-diffusion.

It appears I did have the problem on the latent-diffusion repo as well, but I fixed my problem on both by adding os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "max_split_size_mb:4096" near the start of text2img.py

where exactly should I put it? I tried pasting it in and still got the error granted I have less gpu memory

from stable-diffusion.

It appears I did have the problem on the latent-diffusion repo as well, but I fixed my problem on both by adding os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "max_split_size_mb:4096" near the start of text2img.py

where exactly should I put it? I tried pasting it in and still got the error granted I have less gpu memory

What GPU are you using? Maybe it doesn't have enough VRAM

from stable-diffusion.

entirely possible it's a 6gb GTX 1060. not horrible but not new or anything

from stable-diffusion.

entirely possible it's a 6gb GTX 1060. not horrible but not new or anything

6 GB isn't enough for the large latent diffusion model (which isn't the actual stable diffusion model) . It might be enough for the stable diffusion model once it releases which is half the size.

from stable-diffusion.

I'm still getting the error, even with PYTORCH_CUDA_ALLOC_CONF set.

RuntimeError: CUDA out of memory. Tried to allocate 1.50 GiB (GPU 0; 11.77 GiB total capacity; 8.62 GiB already allocated; 678.19 MiB free; 8.74 GiB reserved in total by PyTorch)

GeForce RTX 3060, 12GB of VRAM

from stable-diffusion.

I solved this by setting --n_samples to 1 and using --n_iter if I wanted more than one output.
Found at: https://www.assemblyai.com/blog/how-to-run-stable-diffusion-locally-to-generate-images/

from stable-diffusion.