Excessive Memory Usage about stand-alone-self-attention HOT 14 OPEN

Holy cow. How did this pass muster? I don't have access to my cluster now but 13.6 gigs for a ResNet-26 is absolutely ridiculous. What's the use of such costly models?

from stand-alone-self-attention.

I have the same problem ResNet-26 on CIFAR-10. Running an RTX 2070 with 8gb graphics card RAM, I get the message:

Tried to allocate 308.00 MiB (GPU 0; 8.00 GiB total capacity; 5.08 GiB already allocated; 266.22 MiB free; 24.83 MiB cached)

This seems really odd. I would also like to know what hardware this was created on.

from stand-alone-self-attention.

I have met the same issue, the memory seems never enough.

from stand-alone-self-attention.

It uses about 13.00 GiB so if your GPU memory is less than that, you may get excessive memory usage problem. Maybe you should reduce the batch size to resolve this.

from stand-alone-self-attention.

13 GB for a ResNet-26?

from stand-alone-self-attention.

I don't know why it is so but this is how it looks like at the moment. Check the GPU Process 1

from stand-alone-self-attention.

I agree with you. This is very costly.

from stand-alone-self-attention.

yes, this is too costly and slow to replace Convs.

from stand-alone-self-attention.

This is ridiculous...I am using p2.16xlarge with 16 GPU and the CUDA memory is still not enough?

from stand-alone-self-attention.

Have you all tried running the experiments with different batch sizes? I was able to run the default CIFAR10 experiments with a batch size of 8 using an RTX 2060. Total memory consumption was under 5 GB. On a more powerful machine with 32 GB V100 cards, I was able to run the default experiment configuration for CIFAR10 while using ~15 GB.

from stand-alone-self-attention.

Have you all tried running the experiments with different batch sizes? I was able to run the default CIFAR10 experiments with a batch size of 8 using an RTX 2060. Total memory consumption was under 5 GB. On a more powerful machine with 32 GB V100 cards, I was able to run the default experiment configuration for CIFAR10 while using ~15 GB.

@iyaja
Hi Ajay, thank you a lot for your reply!
I am running it on the ChestXray dataset. It doesn't work with even a batch size of 1.
Could I ask how did you change the batch size? I do it via --batch-size=1.
What is weird is that the RuntimeError always stays the same no matter how small the batch size I set.
Setting batch size to 32, it says "CUDA out of memory. tried to allocate 25 MB"
Setting batch size to 1, it still says "CUDA out of memory. tried to allocate 25 MB".

from stand-alone-self-attention.

Have you all tried running the experiments with different batch sizes? I was able to run the default CIFAR10 experiments with a batch size of 8 using an RTX 2060. Total memory consumption was under 5 GB. On a more powerful machine with 32 GB V100 cards, I was able to run the default experiment configuration for CIFAR10 while using ~15 GB.

@iyaja
Hi Ajay, thank you a lot for your reply!
I am running it on the ChestXray dataset. It doesn't work with even a batch size of 1.
Could I ask how did you change the batch size? I do it via --batch-size=1.
What is weird is that the RuntimeError always stays the same no matter how small the batch size I set.
Setting batch size to 32, it says "CUDA out of memory. tried to allocate 25 MB"
Setting batch size to 1, it still says "CUDA out of memory. tried to allocate 25 MB".

I also have no idea why it shows gpu_devices: None and gpu: None....Shown in my last image, it seems that all 16 GPUs are occupied.

Another weird thing is... When I was using 8 GPUs, it says CUDA tried to allocate 47MB. When I changed it to 16 GPUs, it says CUDA tried to allocate 25MB....Why increasing GPU just help that little...

from stand-alone-self-attention.

I am trying to use AttentionConv on another model and I got a small Params size but got a super large Forward/backward pass size at the same time.
Why is Forward/backward pass size so large?

from stand-alone-self-attention.

I am trying to use AttentionConv on another model and I got a small Params size but got a super large Forward/backward pass size at the same time. Why is Forward/backward pass size so large?

because it use unfold, it will bring huge gpu memory cost. The param is small... but intermediate featuremaps is very big.

from stand-alone-self-attention.