jongchan / attention-module Goto Github PK

can anyone give me a link to download the pretrained model

Hi, thank you for your codes, it dose help me a lot:)

But there is still some error in the code like
transforms.RandomSizedCrop(size0) in train_imagenet.py
and it seems that you construct your optimizer before you move model parameter to gpus.

Could you please fix this error, thanks for your patience:)

Could you share the model densenet with bam as reported in the paper? Thanks

According to the following code,the W0 is followed by the ReLU activation function? or I misunderstood,cause i'm new to DeepLearning，plz pardon me.
self.mlp = nn.Sequential( Flatten(), nn.Linear(gate_channels, gate_channels // reduction_ratio), #W0 ? nn.ReLU(), nn.Linear(gate_channels // reduction_ratio, gate_channels) #W1 ? )

hello，i would like to ask you that if you have the pre-train model of resnet101+CBAM? i added CBAM module to resnet-101 and trained from scratch. But i have the problem of loss= Nan. Do i need to initialize the paramenters of this module?

look at the ablation study part in the paper

sum gets better performance than prod, maybe you can modify prod to sum.

Originally posted by @Splinter22 in #1 (comment)

What 's the correlation and difference between BAM and CBAM?

Can you provide code to train the VGG model?I have encountered some problems with implementing VGG+CBAM.Thanks~ @Jongchan

It seems that there is an issue in the below line

gate_activation is not defined in code

Thank you for sharing this beautiful work. I have three convolutional layers followed by a global average pooling and then finally a linear layer. Do you recommend placing the BAM / CBAM blocks after each convolutional layer or only after the first two of them? Lastly, do you have a comparison with Gather-Excite? https://arxiv.org/pdf/1810.12348.pdf

Can you share the relevant visual code, thank you

I couldn't get the results of MobileNet that shows on the paper. Could you upload the training code of it? Thank you so much !

Can you please let me know what the "num_layers" parameter is for? and how I should select that? I checked the paper but couldn't find an explanation for that one.

Hello, as you said in paper , the BAM block should be placed in the "bottleneck" place ,and in your pytorch code ,in resnet101 ,there are only 3 BAM Block placed into the backbone, 3 (BAM) 4 (BAM) 6 (BAM) 3 , Can you tell me where to place BAM into mobileNet? Thank you

hello,I can not download the pretrained model! There show 403 Forbidden,how to solve it?Thanks

Hi author, thank for good code. I confused in combine channel and spatial. In your paper before go to sigmoid func, you add two channel, but in code is multiply. Is it mistake?

class BAM(nn.Module):
    def __init__(self, gate_channel):
        super(BAM, self).__init__()
        self.channel_att = ChannelGate(gate_channel)
        self.spatial_att = SpatialGate(gate_channel)
    def forward(self,in_tensor):
        att = 1 + F.sigmoid( self.channel_att(in_tensor) * self.spatial_att(in_tensor) ) # multiply in here
        return att * in_tensor

Hello! When I use bam in mmdetection, I meet
Traceback (most recent call last):
File "./tools/train.py", line 168, in
main()
File "./tools/train.py", line 135, in main
cfg.model, train_cfg=cfg.train_cfg, test_cfg=cfg.test_cfg)
File "/home/gaozishu/data/mmdetection_550/mmdet/models/builder.py", line 67, in build_detector
return build(cfg, DETECTORS, dict(train_cfg=train_cfg, test_cfg=test_cfg))
File "/home/gaozishu/data/mmdetection_550/mmdet/models/builder.py", line 32, in build
return build_from_cfg(cfg, registry, default_args)
File "/usr/local/lib/python3.6/dist-packages/mmcv/utils/registry.py", line 167, in build_from_cfg
return obj_cls(**args)
File "/home/gaozishu/data/mmdetection_550/mmdet/models/detectors/fcos.py", line 17, in init
test_cfg, pretrained)
File "/home/gaozishu/data/mmdetection_550/mmdet/models/detectors/single_stage.py", line 24, in init
self.backbone = build_backbone(backbone)
File "/home/gaozishu/data/mmdetection_550/mmdet/models/builder.py", line 37, in build_backbone
return build(cfg, BACKBONES)
File "/home/gaozishu/data/mmdetection_550/mmdet/models/builder.py", line 32, in build
return build_from_cfg(cfg, registry, default_args)
File "/usr/local/lib/python3.6/dist-packages/mmcv/utils/registry.py", line 167, in build_from_cfg
return obj_cls(**args)
File "/home/gaozishu/data/mmdetection_550/mmdet/models/backbones/mobilenet.py", line 204, in init
self.bam1 = BAM(64 * self.stages)
File "/home/gaozishu/data/mmdetection_550/mmdet/models/backbones/bam.py", line 45, in init
self.channel_att = ChannelGate(gate_channel)
File "/home/gaozishu/data/mmdetection_550/mmdet/models/backbones/bam.py", line 16, in init
gate_channels += [gate_channel // reduction_ratio] * num_layers
TypeError: unsupported operand type(s) for //: 'list' and 'int'

What should I do? Thank you!

Who has experimented with BAM on CIfar100? I can't reach the accuracy of the author. Who can help me? Thanks very much

try

python train_imagenet.py --ngpu 4 --workers 20 --arch resnet --depth 50 --epochs 100 --batch-size 256 --lr 0.1 --att-type BAM --prefix RESNET50_IMAGENET_BAM ./data/ImageNet

showing error

(pytorch) C:\github\attention-module>python train_imagenet.py --ngpu 4 --workers 20 --arch resnet --depth 50 --epochs 100 --batch-size 256 --lr 0.1 --att-type BAM --prefix RESNET50_IMAGENET_BAM ./data/ImageNet
args Namespace(arch='resnet', att_type='BAM', batch_size=256, data='./data/ImageNet', depth=50, epochs=100, evaluate=False, lr=0.1, momentum=0.9, ngpu=4, prefix='RESNET50_IMAGENET_BAM', print_freq=10, resume='', seed=1234, start_epoch=0, weight_decay=0.0001, workers=20)
Traceback (most recent call last):
  File "train_imagenet.py", line 307, in <module>
    main()
  File "train_imagenet.py", line 73, in main
    model = ResidualNet( 'ImageNet', args.depth, 1000, args.att_type )
  File "C:\github\attention-module\MODELS\model_resnet.py", line 200, in ResidualNet
    model = ResNet(Bottleneck, [3, 4, 6, 3], network_type, num_classes, att_type)
  File "C:\github\attention-module\MODELS\model_resnet.py", line 116, in __init__
    self.bam1 = BAM(64*block.expansion)
  File "C:\github\attention-module\MODELS\bam.py", line 45, in __init__
    self.channel_att = ChannelGate(gate_channel)
  File "C:\github\attention-module\MODELS\bam.py", line 12, in __init__
    self.gate_activation = gate_activation
NameError: name 'gate_activation' is not defined

try

python train_imagenet.py --ngpu 4 --workers 20 --arch resnet --depth 50 --epochs 100 --batch-size 256 --lr 0.1 --att-type CBAM --prefix RESNET50_IMAGENET_CBAM ./data/ImageNet

showing error

(pytorch) C:\github\attention-module>python train_imagenet.py --ngpu 4 --workers 20 --arch resnet --depth 50 --epochs 100 --batch-size 256 --lr 0.1 --att-type CBAM --prefix RESNET50_IMAGENET_CBAM ./data/ImageNet
args Namespace(arch='resnet', att_type='CBAM', batch_size=256, data='./data/ImageNet', depth=50, epochs=100, evaluate=False, lr=0.1, momentum=0.9, ngpu=4, prefix='RESNET50_IMAGENET_CBAM', print_freq=10, resume='', seed=1234, start_epoch=0, weight_decay=0.0001, workers=20)
C:\github\attention-module\MODELS\model_resnet.py:129: UserWarning: nn.init.kaiming_normal is now deprecated in favor of nn.init.kaiming_normal_.
  init.kaiming_normal(self.fc.weight)
C:\github\attention-module\MODELS\model_resnet.py:133: UserWarning: nn.init.kaiming_normal is now deprecated in favor of nn.init.kaiming_normal_.
  init.kaiming_normal(self.state_dict()[key], mode='fan_out')
Traceback (most recent call last):
  File "train_imagenet.py", line 307, in <module>
    main()
  File "train_imagenet.py", line 81, in main
    model = torch.nn.DataParallel(model, device_ids=list(range(args.ngpu)))
  File "C:\Users\admin\.conda\envs\pytorch\lib\site-packages\torch\nn\parallel\data_parallel.py", line 131, in __init__
    _check_balance(self.device_ids)
  File "C:\Users\admin\.conda\envs\pytorch\lib\site-packages\torch\nn\parallel\data_parallel.py", line 18, in _check_balance
    dev_props = [torch.cuda.get_device_properties(i) for i in device_ids]
  File "C:\Users\admin\.conda\envs\pytorch\lib\site-packages\torch\nn\parallel\data_parallel.py", line 18, in <listcomp>
    dev_props = [torch.cuda.get_device_properties(i) for i in device_ids]
  File "C:\Users\admin\.conda\envs\pytorch\lib\site-packages\torch\cuda\__init__.py", line 301, in get_device_properties
    raise AssertionError("Invalid device id")
AssertionError: Invalid device id

What should I do?

The url for pre-trained model doesnt seem to work right now.
Can you please provide the model again?

Thank you

In BAM.py : in class ChannelGate, I think self.gate_activation=gate_activation is error, and it should be self.gate_channel=gate_channel ?
Looking forward to your answer！！

Thanks for sharing the code in the PyTorch version.

I try to use the model_resnet.py and notice that it can work on the CPU. But when testing it on GPU, I always get an error "IndexError: Dimension out of range (expected to be in the range of [-2, 1], but got 2)" in the line "F.sigmoid(channel_att_sum).unsqueeze(2).unsqueeze(3).expand_as(x)".

I would like to ask how to solve this problem. Thanks.

Hello,

Thank you for your great work.
The link to the pre-trained model did not seem to work right now.
Can you please provide the trained model again?

Thank you and best regards.

In BAM paper, element-wise summation achieves the best performance.
But element-wise product is used in bam.py.
Is there any special reason why you get element-wise product?

I noticed that there are extra pooling-types in the class Channelgate, i.e. , lp_pool and lse_pool. Are these your new attempt? And I'm wondering if these methods are effective?

class ChannelPool(nn.Module): def forward(self, x): return torch.cat( (torch.max(x,1)[0].unsqueeze(1), torch.mean(x,1).unsqueeze(1)), dim=1 )
torch.max(x,1)[0] is very strange here.
Is that a bug?
Why we need get the feature map of the first image?
@Jongchan

Hello! I'm gonna try CBAM. But when I looked at the code, I was curious.
The locations where BAM and CBAM are applied are different in your resnet code(example code). Could you tell me what is diffrence?

Hi,
in your paper you say you are choosing element-wise summation to combine the two branches, however in your code you are using a product

Could you elaborate on this?
Thanks

Hi,
Thank you for sharing your work. I have a problem about how to visualization the attention. Thanks

I think there is possibly an inconsistency in the location of the application of the Batch Normalization between the paper and the code.

1st instance of possible inconsistency:

The paper mentions on Page 4, Section 3, subsection Channel attention branch that:

So it is understod that the batch norm is applied at the end of the MLP, i.e. after the final layer. However, from the implementation, shows final layer in forward() function as:

Where self.gate_c's final layer is defined as:

When I tried, below error occurred.

Traceback (most recent call last):
File "train.py", line 310, in
main()
File "train.py", line 149, in main
train(train_loader, model, criterion, optimizer, epoch)
File "train.py", line 193, in train
losses.update(loss.data[0], input.size(0))
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

Is there anyone who tried to train with own custom dataset?

i am troubled how to use grad cam in se-nets, for there is not a conv layer to show like others.

At first, thanks you for your code.
I want to train the BAM Model on the CIFAR-100 dataset, but I can't find a series of parameters configuration about using the CIFAR-100 dataset.
Can you provide the train_cifar100.py, similar to the "train_imagenet.py"
Thanks again!

Hi, I am working on a classification of 3D MRI where I want to combine a mask and a raw MRI and I was wondering if someone has implemented those models on 3D images

Hi, I download the ResNet50+CBAM trained model. In the .pth file, I found the key:value is that:
'best_prec1':0
'epoch':-1
It is the right trained model? Thanks!

in BAM ChannelGate:

this line assumes the input H=W

to fix this, on can change the code into:

avg_pool = F.avg_pool2d( in_tensor, (in_tensor.size(2), in_tensor.size(3)), stride=in_tensor.size(2) )

I try this module in detection task. When i try to make a inference, it throw an error. I found that it can not run forward in one batch size because of the nn.BatchNorm1d . How to deal with it?

dear author:
in your model_resnet.py there is a cruicial issue which need to be corrected!
"""
if network_type == "ImageNet":
...
self.avgpool = nn.AvgPool2d(7)
"""
should be modified below:
"""
if network_type == "ImageNet":
...
self.avgpool = nn.AvgPool2d(7, stride=1)
"""

I want to use your proposed model ResNet50+CBAM as backbone in another model on different datasets. Please guide me how I can use it and share detailed steps at least. Thank you

I download the pretrained model of the BAM, but when I depcompress it, the error occured.
tar -xvf RESNET50_IMAGENET_BAM_best.pth.tar
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
could you update it? Thank you!

How to run this in keras
I mean how to / where to find for keras equivalent of these codes

I added CBAM and BAM in ResNet50 on fine grained dataset, e.g. CUB200. But it had a worse performance than pure ResNet50 backbone. Was it because I failed to have right training hyper parameters? Its also possible that the two methods are only effective for general classification but not suitable for fine grained classification. Dose anyone have the same experience or some suggestions? Thx!

I think it is weird, can you explain that?

How to conbine two attention modules "in parallel" ?