fwang91 / residual-attention-network Goto Github PK

According to your paper, the "Interp" layer does a bilinear interpolation to upsample its previous layer's output. But "Interp" layer has two inputs in your implementation. I'm not very familiar with caffe. Could you provide some documentations of "Interp" layer in caffe? Is there any alternatives in tensorflow or pytorch?

Hi,
I wanted to ask how to visualize the attention masks as given in Fig 1 in the paper? Does it involve using GRAD CAM? or is it directly the actual outputs of the mask? Also, given that the masks are of different sizes and the number of channels are more (usually 256, 512 or 1024), how do the authors visualize the attention masks then?
A code snippet would help highly.

in your residual-attention-network caffe , the end of interp.cpp is :

INSTANTIATE_CLASS(InterpLayer);
//REGISTER_LAYER_CLASS(InterpLayer);

which means you comment the InterpLayer;

while in your provided caffe（https://github.com/fwang91/caffe/blob/master/src/caffe/layers/interp.cpp）code， the end of interp.cpp is :

INSTANTIATE_CLASS(InterpLayer);
REGISTER_LAYER_CLASS(InterpLayer);

which means you uncomment the InterpLayer;

i compile the residual-attention-network version caffe, it can make ok, but is that means you do not use InterpLayer in residual-attention-network ?

BTW, when i compile your provided caffe with REGISTER_LAYER_CLASS(InterpLayer); it shows the following error:

src/caffe/layers/interp.cpp:111:1: note: in expansion of macro 'REGISTER_LAYER_CLASS'
REGISTER_LAYER_CLASS(InterpLayer);
^
src/caffe/layers/interp.cpp:111:22: error: 'InterpLayerLayer' does not name a type
REGISTER_LAYER_CLASS(InterpLayer);
^
./include/caffe/layer_factory.hpp:135:42: note: in definition of macro 'REGISTER_LAYER_CLASS'
return shared_ptr<Layer >(new type##Layer(param));
^
./include/caffe/layer_factory.hpp:135:59: error: expected primary-expression before '>' token
return shared_ptr<Layer >(new type##Layer(param));
^
src/caffe/layers/interp.cpp:111:1: note: in expansion of macro 'REGISTER_LAYER_CLASS'
REGISTER_LAYER_CLASS(InterpLayer);
^
Makefile:581: recipe for target '.build_release/src/caffe/layers/interp.o' failed

can you show me some tips about why in your residual-attention-network, you comment the REGISTER_LAYER_CLASS(InterpLayer), while in your provided caffe, once umcomment the REGISTER_LAYER_CLASS(InterpLayer), it show errors ?

Hi, I would like to know how to plot the feature map with attention, similar in your paper. Would you like to share the code about how to plot those figures? Thanks a lot.

Thanks.

make: *** [.build_release/cuda/src/caffe/util/interp.o] 错误 1
make: *** 正在等待未完成的任务....
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).

hello, when I make caffe, I meet this problem

./include/caffe/common.cuh(9): error: function "atomicAdd(double *, double)" has already been defined

1 error detected in the compilation of "/tmp/tmpxft_000040a6_00000000-11_interp.compute_61.cpp1.ii".
Makefile:594: recipe for target '.build_release/cuda/src/caffe/util/interp.o' failed
make: *** [.build_release/cuda/src/caffe/util/interp.o] Error 1
make: *** Waiting for unfinished jobs....

could you tell me how to fix it?

Please tell me how about the settings in BN layers in training phase.
think you!

Hi, i am intereated in your work and want to fine-tune the model for other task, could you please share the pre-trained imagenet model?

import caffe
affe.set_mode_gpu()

net = caffe.Net("/path-to/Attention-56-dcase.prototxt",caffe.TRAIN)

got error:
I1207 11:37:31.168298 31752 layer_factory.hpp:77] Creating layer conv1/bn
F1207 11:37:31.168318 31752 layer_factory.hpp:81] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown layer type: BN (known types: AbsVal, Accuracy, ArgMax, BNLL, BatchNorm, BatchReindex, Bias, Concat, ContrastiveLoss, Convolution, Crop, Data, Deconvolution, Dropout, DummyData, ELU, Eltwise, Embed, EuclideanLoss, Exp, Filter, Flatten, HDF5Data, HDF5Output, HingeLoss, Im2col, ImageData, InfogainLoss, InnerProduct, Input, Interp, LRN, LSTM, LSTMUnit, Log, MVN, MemoryData, MultinomialLogisticLoss, PReLU, Parameter, Pooling, Power, RNN, ReLU, Reduction, Reshape, SPP, Scale, Sigmoid, SigmoidCrossEntropyLoss, Silence, Slice, Softmax, SoftmaxWithLoss, Split, TanH, Threshold, Tile, WindowData)
*** Check failure stack trace: ***

Thanks for your great job! I have a question about the expression of mix attention and how to implement it. And is conv->relu->conv->sigmoid able to represent?

Hello,

I was wondering if we can have the solvers you use to train your networks.
Thanks!

Hi,

Many thanks for your excellent works! I want to train your network, can you please release the train prototxt?

Best,

You mentioned in the paper that the batch size for training is 32. However for my GPU which has 8G I set the batch size to 8 and almost all the memory is used.

So I am wondering how much memory your GPU has?

Thanks.

Interested in your paper, I find some bugs when I compile your caffe. My environment is Centos 7.5, Cuda 8.0, Cudnn5.1.
The following is the fix:

diff --git a/include/caffe/common.cuh b/include/caffe/common.cuh
index 63e7399..4ce4a40 100644
--- a/include/caffe/common.cuh
+++ b/include/caffe/common.cuh
@@ -6,16 +6,20 @@
 #include <cuda.h>
 
 // CUDA: atomicAdd is not defined for doubles
-static __inline__ __device__ double atomicAdd(double *address, double val) {
-  unsigned long long int* address_as_ull = (unsigned long long int*)address;
-  unsigned long long int old = *address_as_ull, assumed;
-  if (val==0.0)
-    return __longlong_as_double(old);
-  do {
-    assumed = old;
-    old = atomicCAS(address_as_ull, assumed, __double_as_longlong(val +__longlong_as_double(assumed)));
-  } while (assumed != old);
-  return __longlong_as_double(old);
-}
+    #if !defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 600
+
+    #else
+    static __inline__ __device__ double atomicAdd(double *address, double val) {
+      unsigned long long int* address_as_ull = (unsigned long long int*)address;
+      unsigned long long int old = *address_as_ull, assumed;
+      if (val==0.0)
+        return __longlong_as_double(old);
+      do {
+        assumed = old;
+        old = atomicCAS(address_as_ull, assumed, __double_as_longlong(val +__longlong_as_double(assumed)));
+      } while (assumed != old);
+      return __longlong_as_double(old);
+    }
+    #endif
 
 #endif
diff --git a/src/caffe/layers/bn_layer.cpp b/src/caffe/layers/bn_layer.cpp
index 4b6ba1a..b530ac8 100644
--- a/src/caffe/layers/bn_layer.cpp
+++ b/src/caffe/layers/bn_layer.cpp
@@ -318,4 +318,5 @@ namespace caffe {
 #endif
 
   INSTANTIATE_CLASS(BNLayer);
+  REGISTER_LAYER_CLASS(BN);
 }  // namespace caffe
diff --git a/src/caffe/layers/interp.cpp b/src/caffe/layers/interp.cpp
index 02a87cf..d9200e2 100644
--- a/src/caffe/layers/interp.cpp
+++ b/src/caffe/layers/interp.cpp
@@ -108,6 +108,6 @@ STUB_GPU(InterpLayer);
 
 
 INSTANTIATE_CLASS(InterpLayer);
-//REGISTER_LAYER_CLASS(InterpLayer);
+REGISTER_LAYER_CLASS(Interp);
 
 }  // namespace caffe