Hardware Requirement, does eval need that 12G GPU memory? about bottom-up-attention HOT 18 CLOSED

Guys, the features extraction works on cpu!

GPU 3: 1/108067 91.213s (projected finish: 2738.10 hours) # 113 days

from bottom-up-attention.

From a computational perspective, the size of the input image doesn't matter as they are all resized to 600 pixels on the shortest side.

The easiest way to reduce the memory requirement is to reduce the SCALES and MAX_SIZE image parameters at test time. The parameters are defined here but you can set the values in this config file.

This might reduce the quality / number of features a bit, so a better option would be to maybe chop the network in half after the RPN layers, then run it in two stages (saving intermediate outputs). This would require modifying the code and network though.

Hope that helps.

from bottom-up-attention.

12g is enough.
as anderson said, i extract feature by run it in two stage.
split models/vg/ResNet-101/faster_rcnn_end2end_final/train.prototxt to
models/vg/ResNet-101/faster_rcnn_end2end_final/rpn.prototxt
models/vg/ResNet-101/faster_rcnn_end2end_final/rcnn.prototxt
and modify some code in tools/generate_tsv.py
good luck

from bottom-up-attention.

i have gpu 1080(8G memory)x4, is there anyway i can use multi gpu when predict?

from bottom-up-attention.

cpu mode running success too

from bottom-up-attention.

for generate feature, we need 12G per GPU?

from bottom-up-attention.

any idea to generate feature with 8G GPU x4 on private image data, cpu is a long long time......

from bottom-up-attention.

thx!

from bottom-up-attention.

@StephenZengCn how to solve this problem finally? I come across the same problem, and my gpu has 12g memory. Thanks.

from bottom-up-attention.

@StephenZengCn Thanks for your reply. I'm new to caffe. So it is possible for your to share the modifications to run the demo.ipynb? Thanks a lot!

from bottom-up-attention.

i didn't use demo.ipynb, just use generate_tsv.py to extract bottom-up features.

from bottom-up-attention.

@yangshao did u run demo.ipnb successfully ?

from bottom-up-attention.

12g is enough.
as anderson said, i extract feature by run it in two stage.
split models/vg/ResNet-101/faster_rcnn_end2end_final/train.prototxt to
models/vg/ResNet-101/faster_rcnn_end2end_final/rpn.prototxt
models/vg/ResNet-101/faster_rcnn_end2end_final/rcnn.prototxt
and modify some code in tools/generate_tsv.py
good luck

Hi @StephenZengCn ,

I am also facing the out-of-memory problem when trying to extract bottom-up features. Could you please share the detail (or these files) of how you split the network?

Thank you!

from bottom-up-attention.

Hi @StephenZengCn

Could you please share the scripts of them :)? I would really appreciate for the help.

from bottom-up-attention.

Does anyone have some no grad tricks like Variable(data, volatile=True) in Pytorch, which doing forward calculation only rather than saving blobs for each layer.

from bottom-up-attention.

@coldmanck : I faced the same issue and the model really needed close to 10.5 gb of memory on a GPU. So I made sure that I have one with enough memory.

from bottom-up-attention.

I also met this problem when using 11g GPU. How I can solve it?

from bottom-up-attention.

I use this code to extract feature from VG datasets, and I found it was caused by some images with big difference in aspect ratio, for example 281 * 500, faster rcnn will scale based on the shorter one, so making the larger one to large.

this code may work on 11G gpu

from bottom-up-attention.