dingxiaoh / acnet Goto Github PK

View Code? Open in Web Editor NEW

821.0 821.0 133.0 203 KB

ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks

License: MIT License

Python 100.00%

acnet's People

Contributors

Stargazers

Watchers

Forkers

soonhwan-kwon mrzhouyang ww2401 seeker1943 chaos1992 chaoso chrisyxue richardych banyueqin snowhou zhaotudou mahlermozart ayetao c1a1o1 wolfworld6 ansj11 zhaojp-frank shaochengjia zhwzhong robot-ai-machinelearning maxtian666 chenmingthu frizy-up wrrjasmine limeng0307 jlhou suyanzhou626 keyky thecryboy thebenchmarkh andyliu93 lxy6 water2bear dukedj hxl1990 mengkunzhao quqizhe-whu 2017tjm zclonely milkigit huster123 zhumingxu hanshuobest bazingaauv igi123 wxphb zhzhuangxue surej0824 hanj2270 kwan96 txytju chinadavid1991 lovecodestudent houhaopeng boy-new-2019 suzoosuagr hell-to-heaven zcl912 lchenxidian tonylibing liu-ca guidewsp minygd whitecosmos1996 tengerye mtcai yukaizhou gdy1201 jstar-s yutpa yangliuqing-add vinceyzw kasyoukin xrosliang ballic zzzhoudj wen0618 wangshouwen shenzheji cv-ip binianzjl zyg11 jefterf simenglv qiuhuan zcc1019 jinming0912 sysu-shey zeta1999 dafuny yhjflower xxyu2012 icemanjia doctor-damu lufeng22 scotter-qian ericlong423 showlo qgh1223 luxdong

acnet's Issues

train loss and accuracy curve for single gpu

in the programme named ndp_train.py. When I try to draw a train loss curve and accuracy curve with a single gpu(the network just contains 6 layers).I realize that the prerequisites(if iteration % cfg.tb_iter_period == 0 and engine.world_rank == 0) I can not achieve. because you have set the engine.world_rank == 1 when training a net with a single gpu( self.world_size = 1,self.world_rank =
,self.devices = [0]). I am very confused. if training loss in a single gpu should not draw a related curve as a criterion

Performance if we put the BN after merging the branches

Hi, thanks for your great work.
After reading your paper, I wonder how does the ACNet perform if we put the BN layer after merging the branches and remove the BN layer in each branch. There may be some degradation. Have you conducted any experiments?

绘制曲线

@DingXiaoH @ShawnDing1994
请问作者，论文里面figure5的图是怎么画的，想学习一下。是怎么画出带不同颜色的阴影的对应曲线的？谢谢

when will release ImageNet pretrain model?

预训练权重，这个到哪里下载

关于ACBlock部分逻辑的疑问

您好，拜读您的论文及开源代码后，有一些疑问想向您咨询。

acb.py Line21

center_offset_from_origin_border = padding - kernel_size // 2
上述语句是在计算什么，能否简单阐述一下
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

acb.py Line24~Line33

if center_offset_from_origin_border >= 0:
……
else:
……
这里面的两个分支分别代表什么？if分支比较好理解，一些通常的设定例如p=1，k=3均会进入该分支，但是else分支对应哪些具体的情况？
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

acb.py Line29~Line33

else:
self.ver_conv_crop_layer = CropLayer(crop_set=ver_pad_or_crop)
ver_conv_padding = (0, 0)
self.hor_conv_crop_layer = CropLayer(crop_set=hor_pad_or_crop)
hor_conv_padding = (0, 0)

若进入else分支，则center_offset_from_origin_border<0,该分支下执行对输入图的crop操作，
#crop_layer.py
class CropLayer(nn.Module):

E.g., (-1, 0) means this layer should crop the first and last rows of the feature map. And (0, -1) crops the first and last columns

def init(self, crop_set):
super(CropLayer, self).init()
self.rows_to_crop = - crop_set[0]
self.cols_to_crop = - crop_set[1]
assert self.rows_to_crop >= 0
assert self.cols_to_crop >= 0

def forward(self, input):
if self.rows_to_crop == 0 and self.cols_to_crop == 0:
return input
elif self.rows_to_crop > 0 and self.cols_to_crop == 0:
return input[:, :, self.rows_to_crop:-self.rows_to_crop, :]
elif self.rows_to_crop == 0 and self.cols_to_crop > 0:
return input[:, :, :, self.cols_to_crop:-self.cols_to_crop]
else:
return input[:, :, self.rows_to_crop:-self.rows_to_crop, self.cols_to_crop:-self.cols_to_crop]

显然一旦进入else分支，以crop_set=ver_pad_or_crop为例，crop_set=(padding, center_offset_from_origin_border)，即crop_set[1]<0,即 self.cols_to_crop = - crop_set[1]>0,永远不可能出现self.cols_to_crop == 0的情况，为何forward中仍出现相关的判断情况。

此外，若p=1,k=5,显然也是进入else分支的，此时crop_set=ver_pad_or_crop=（1，-1）
那么self.rows_to_crop = - crop_set[0]=-1，由语句assert self.rows_to_crop >= 0触发报错，显然，只要进入else分支时p不等于0均会产生该问题，很让人困惑。

既然明确了
assert self.rows_to_crop >= 0
assert self.cols_to_crop >= 0

为什么在forward中仍然要下面这种判断情况，我认为由上面的断言存在，不可能会有机会进入这个分支
else:
return input[:, :, self.rows_to_crop:-self.rows_to_crop, self.cols_to_crop:-self.cols_to_crop]
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
希望您能在百忙中回复，感谢！

About params and flops

Hi author @ShawnDing1994 @DingXiaoH,

That's a milestone work on ACBlock, it's really impressive!

I have read the paper that you mentioned it will not generate extra computational burdens. But only based on three times convolution (square, vertical and horizontal), the MACC will absolutely increase. Did you calculate the FLOPs and # of params on some popular models applied ACBlock, eg. ResNet?

Sorry for my misunderstand in advance if I'm wrong.

ResNet56

Hello, which file caotains the code of ResNet56 that use AC-Block?

Can I directly replace conv2d in CNN with acblock

Error in ACBuilder

@DingXiaoH
我想测一下作者的resnet，但是得到了以下报错，请问是怎么回事？

self.conv1 = builder.Conv2dBNReLU(3, int(64 * width_multiplier), kernel_size=3, stride=1, padding=1)
TypeError: Conv2dBNReLU() missing 1 required positional argument: 'out_channels'

请问有TensorFlow1.x的版本吗？

pytorch和tf2的框架都很简单，ACB也好实现，tf1.x的使用相对复杂，请问有tf1.x的版本吗

my data

你好，我想训练自己的数据库需要怎么做？

Hello, what do I need to do to train my own database?

this work is great, but can you give a example how we can replace 33 conv by using 33+13+31

How about constructing an ACB with several different dxd kernels?

Will the performance be improved further?

Different result?

Hi. I have downloaded your github code and trained the ResNet-56 on cifar10. But the result is totally different from your paper.
In the paper, ResNet-56 trained on cifar10:
Model Base Top-1 ACNet Top-1
ResNet-56 94.31 95.09

But in my experiment,

python acnet_rc56.py --try_arg=normal_lrs1

INFO validate at epoch 496, top1=93.80000, top5=99.80000, loss=0.320561.80000, top5=99.80000, loss=0.32056

python acnet_rc56.py --try_arg=acnet_lrs1

validate at epoch 492, top1=91.70000, top5=99.75000, loss=0.442718.70000, top5=99.75000, loss=0.44272

My pytorch version:
torch==1.2.0
torchvision==0.4.0

You would see in my experiment the ACNet result is worse than normal training and I have done it twice. The results have no change.

can not find the models

In your TUDOS, I did not find the following models :
from acnet import AsymConvBlock, acnet_fuse_and_load, acnet_switch_to_deploy
Where are they?
Can you introduce the function of them ,please?

TUDOS:
from acnet import AsymConvBlock, acnet_fuse_and_load, acnet_switch_to_deploy

build model, replace regular Conv2d with AsymConvBlock

class YourNet(nn.module):
...
self.conv1 = AsymConvBlock(in_channels=..., out_channels=..., ...)
self.conv2 = AsymConvBlock(in_channels=..., out_channels=..., ...)

train

model = YourNet(...)
train(model)
model.save_checkpoint(SAVE_PATH) # use just the same PyTorch functions

deploy

acnet_switch_to_deploy()
deploy_model = YourNet(...) # here deploy_model should be of the same structure as baseline
acnet_fuse_and_load(deploy_model, SAVE_PATH) # use the converted weights to initliaze it
test(model)

how to deploy the trained model ？

I had try to add the ACBlock which in acb.py into my pytorch model and trained. it looks good maybe?... But i dont know how to deploy it .It seems should be convert it ,for example ,use the ancnet_fussion.py .But i try and guess it maybe a keras model with h5py. Emm...idont know how to handle it.

question about the behavior of bn layers in inference

Hi,

Thanks for bring this great work to the community !! There is something that I find I cannot make clear after reading the paper.

In the paper, you mentioned that in each of the AC branch, there will be a bn layer after the convolution layers (square or asymmetric). In the inference phase, the weights of the conv layers are added up with the asymmetric conv weights added to the skeleton part of the square conv weights. However, I do not understand how the bn layers are fused in the inference. After going through the code, I find that you seems to fuse the bn layers to conv layers, thus there will not be bn layers in the model in the inference phase. If we do not fuse the bn layers to convolution layers, how should we deal with the bn layers in the three branches? Should I simply summed up the weights and running states be make a single bn layer?

请问可以共享一下imagnet训练集的预训练权重吗？

Does the val loss match the real performance of the inference model?

Hi, I am trying to ensemble your work into my detection network,
And I have some confusion about the process of validation. During inference, the network would be modified to merge the kernel, but it seems that the network remain the same as training in the process of validation during training.
I am wondering whether the val loss could actually match the real performance with the inference model.
I will appreciate it if someone reply, thank you!

Good work but more analysis of acnet could be done

I think this is some interesting work. In theory, it feels like the 1x3 and 3x1 kernels essentially just increase the augment the gradient of the skeleton entries compared to a single 3x3 kernel.

Another difference is then the initialization.

ACnet should also converge faster than standard nets, since now we are doing multiple weight initialization for skeleton entries, it has a better chance to produce a weight that is closer to a local minimum.

This perhaps helps with escaping local saddle points as well, since now the optimizer can descend from multiple angles for the skeleton entries.

I think at least another paper for a much better analysis on the theory side can be done :)

Release a PyTorch module

will you tell me when the Release a PyTorch module is available?

what about use dw-conv instead of conv in ACBlock?

nice work,
@DingXiaoH have u done some experiments of replacing conv with dw-conv?

I know why there are so few stars.

The author is soooo lazy

depth conv的实验

请问使用depth conv的mobilenet支持这种权重融合吗

TypeError problem

return super(ACNetBuilder, self).Conv2dBN(in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size, stride=stride,
TypeError: super(type, obj): obj must be an instance or subtype of type

when I replace previous conv2d with the new Conv2d or Conv2dBN, this TypeError occur. Have you ever met this problem . Can you tell me how to solve this ? thankx

Is the difference from the original paper only in the learning rate?

The result here is 94.31 with resnet-56 for cifar-10, but in the original resnet paper from He, the result is 93.03.
Is the difference only in the learning rate?

验证时如何把推理转换后的模型变回去

ACNet reimplement based on PaddlePaddle

Hi, we re-implemented ACNet based on PaddlePaddle, which is included in official models.

models

weight utils

Nice work to ACNet!

Fail to Produce ResNet 56 （w/o acblock）on cifar 10， on get 94.0x%

Here is the training log

Start training with pytorch version 1.5.0
2020-07-06 19:20:50,065 train INFO: num of params in hdf5=860083
2020-07-06 19:20:54,002 train INFO: validate at epoch 0, top1=10.00000, top5=50.00000, loss=57939871911518302401200128.000000
2020-07-06 19:21:03,325 train INFO: validate at epoch 222, top1=93.89000, top5=99.78000, loss=0.289868
2020-07-06 19:21:40,101 train INFO: validate at epoch 1, top1=36.57000, top5=86.52000, loss=1.771166
2020-07-06 19:21:49,070 train INFO: validate at epoch 223, top1=93.81000, top5=99.79000, loss=0.292585
2020-07-06 19:22:25,864 train INFO: validate at epoch 2, top1=47.88000, top5=93.74000, loss=1.402634
2020-07-06 19:22:35,428 train INFO: validate at epoch 224, top1=93.73000, top5=99.74000, loss=0.295463
2020-07-06 19:23:11,434 train INFO: validate at epoch 3, top1=51.99000, top5=93.54000, loss=1.390009
2020-07-06 19:23:21,190 train INFO: validate at epoch 225, top1=93.70000, top5=99.79000, loss=0.294498
2020-07-06 19:23:56,947 train INFO: validate at epoch 4, top1=63.18000, top5=96.63000, loss=1.072672
2020-07-06 19:24:07,093 train INFO: validate at epoch 226, top1=93.64000, top5=99.76000, loss=0.297798
2020-07-06 19:24:40,152 train INFO: validate at epoch 5, top1=70.15000, top5=97.39000, loss=0.877936
2020-07-06 19:24:50,742 train INFO: validate at epoch 227, top1=93.74000, top5=99.76000, loss=0.298641
2020-07-06 19:25:23,892 train INFO: validate at epoch 6, top1=69.92000, top5=96.88000, loss=0.908797
2020-07-06 19:26:02,437 train INFO: validate at epoch 7, top1=68.23000, top5=97.72000, loss=1.011845
2020-07-06 19:26:41,294 train INFO: validate at epoch 8, top1=73.72000, top5=98.46000, loss=0.824451
2020-07-06 19:27:20,177 train INFO: validate at epoch 9, top1=79.14000, top5=98.90000, loss=0.616009
2020-07-06 19:27:58,551 train INFO: validate at epoch 10, top1=77.63000, top5=98.57000, loss=0.681937
2020-07-06 19:28:37,283 train INFO: validate at epoch 11, top1=81.24000, top5=99.12000, loss=0.545091
2020-07-06 19:29:15,623 train INFO: validate at epoch 12, top1=79.25000, top5=98.88000, loss=0.636652
2020-07-06 19:29:52,386 train INFO: validate at epoch 13, top1=79.15000, top5=98.54000, loss=0.642556
2020-07-06 19:30:31,386 train INFO: validate at epoch 14, top1=80.31000, top5=99.09000, loss=0.582592
2020-07-06 19:31:10,495 train INFO: validate at epoch 15, top1=80.22000, top5=99.00000, loss=0.578624
2020-07-06 19:31:49,522 train INFO: validate at epoch 16, top1=81.21000, top5=98.92000, loss=0.566737
2020-07-06 19:32:28,693 train INFO: validate at epoch 17, top1=78.98000, top5=99.20000, loss=0.660530
2020-07-06 19:33:07,775 train INFO: validate at epoch 18, top1=81.60000, top5=99.24000, loss=0.545860
2020-07-06 19:33:46,862 train INFO: validate at epoch 19, top1=80.32000, top5=98.82000, loss=0.682879
2020-07-06 19:34:23,413 train INFO: validate at epoch 20, top1=82.50000, top5=99.32000, loss=0.525634
2020-07-06 19:35:02,580 train INFO: validate at epoch 21, top1=81.35000, top5=98.80000, loss=0.592682
2020-07-06 19:35:41,674 train INFO: validate at epoch 22, top1=84.74000, top5=99.18000, loss=0.466926
2020-07-06 19:36:20,841 train INFO: validate at epoch 23, top1=74.90000, top5=98.35000, loss=0.872360
2020-07-06 19:36:59,477 train INFO: validate at epoch 24, top1=83.97000, top5=99.25000, loss=0.485715
2020-07-06 19:37:38,633 train INFO: validate at epoch 25, top1=83.40000, top5=99.19000, loss=0.513213
2020-07-06 19:38:17,300 train INFO: validate at epoch 26, top1=83.92000, top5=98.79000, loss=0.518512
2020-07-06 19:38:55,480 train INFO: validate at epoch 27, top1=82.48000, top5=99.10000, loss=0.539502
2020-07-06 19:39:31,240 train INFO: validate at epoch 28, top1=86.64000, top5=99.53000, loss=0.414578
2020-07-06 19:40:01,934 train INFO: validate at epoch 29, top1=86.30000, top5=99.38000, loss=0.408653
2020-07-06 19:40:32,141 train INFO: validate at epoch 30, top1=84.02000, top5=99.35000, loss=0.511296
2020-07-06 19:41:02,734 train INFO: validate at epoch 31, top1=84.54000, top5=99.39000, loss=0.484771
2020-07-06 19:41:32,942 train INFO: validate at epoch 32, top1=85.45000, top5=99.29000, loss=0.432758
2020-07-06 19:42:03,549 train INFO: validate at epoch 33, top1=84.97000, top5=99.17000, loss=0.482988
2020-07-06 19:42:35,369 train INFO: validate at epoch 34, top1=85.45000, top5=99.29000, loss=0.484890
2020-07-06 19:43:14,032 train INFO: validate at epoch 35, top1=85.63000, top5=99.37000, loss=0.472127
2020-07-06 19:43:52,719 train INFO: validate at epoch 36, top1=81.45000, top5=98.90000, loss=0.641560
2020-07-06 19:44:30,434 train INFO: validate at epoch 37, top1=86.22000, top5=99.41000, loss=0.444391
2020-07-06 19:45:01,086 train INFO: validate at epoch 38, top1=84.14000, top5=99.24000, loss=0.568259
2020-07-06 19:45:31,705 train INFO: validate at epoch 39, top1=86.39000, top5=99.60000, loss=0.438984
2020-07-06 19:46:02,600 train INFO: validate at epoch 40, top1=86.60000, top5=99.50000, loss=0.450507
2020-07-06 19:46:33,288 train INFO: validate at epoch 41, top1=86.80000, top5=99.54000, loss=0.436938
2020-07-06 19:47:03,475 train INFO: validate at epoch 42, top1=87.20000, top5=99.56000, loss=0.435762
2020-07-06 19:47:33,950 train INFO: validate at epoch 43, top1=86.09000, top5=99.50000, loss=0.464236
2020-07-06 19:48:04,670 train INFO: validate at epoch 44, top1=86.03000, top5=99.46000, loss=0.459956
2020-07-06 19:48:34,963 train INFO: validate at epoch 45, top1=87.45000, top5=99.39000, loss=0.426245
2020-07-06 19:49:05,110 train INFO: validate at epoch 46, top1=87.58000, top5=99.64000, loss=0.391867
2020-07-06 19:49:35,520 train INFO: validate at epoch 47, top1=86.77000, top5=99.33000, loss=0.437984
2020-07-06 19:50:06,224 train INFO: validate at epoch 48, top1=84.51000, top5=98.80000, loss=0.582838
2020-07-06 19:50:37,062 train INFO: validate at epoch 49, top1=85.53000, top5=99.27000, loss=0.501077
2020-07-06 19:51:07,504 train INFO: validate at epoch 50, top1=85.95000, top5=99.41000, loss=0.470488
2020-07-06 19:51:47,854 train INFO: validate at epoch 51, top1=85.82000, top5=99.14000, loss=0.489779
2020-07-06 19:52:26,613 train INFO: validate at epoch 52, top1=87.92000, top5=99.44000, loss=0.395293
2020-07-06 19:53:05,288 train INFO: validate at epoch 53, top1=84.62000, top5=99.50000, loss=0.537296
2020-07-06 19:53:44,526 train INFO: validate at epoch 54, top1=87.74000, top5=99.71000, loss=0.414536
2020-07-06 19:54:23,649 train INFO: validate at epoch 55, top1=88.59000, top5=99.62000, loss=0.372694
2020-07-06 19:55:02,947 train INFO: validate at epoch 56, top1=88.12000, top5=99.50000, loss=0.413502
2020-07-06 19:55:42,082 train INFO: validate at epoch 57, top1=87.62000, top5=99.56000, loss=0.418841
2020-07-06 19:56:19,827 train INFO: validate at epoch 58, top1=86.87000, top5=99.19000, loss=0.456641
2020-07-06 19:56:58,817 train INFO: validate at epoch 59, top1=88.04000, top5=99.45000, loss=0.421818
2020-07-06 19:57:37,877 train INFO: validate at epoch 60, top1=87.28000, top5=99.61000, loss=0.426081
2020-07-06 19:58:17,332 train INFO: validate at epoch 61, top1=88.33000, top5=99.60000, loss=0.382688
2020-07-06 19:58:56,661 train INFO: validate at epoch 62, top1=87.45000, top5=99.44000, loss=0.441726
2020-07-06 19:59:35,714 train INFO: validate at epoch 63, top1=87.74000, top5=99.51000, loss=0.425312
2020-07-06 20:00:14,432 train INFO: validate at epoch 64, top1=87.86000, top5=99.59000, loss=0.391461
2020-07-06 20:00:53,471 train INFO: validate at epoch 65, top1=88.66000, top5=99.57000, loss=0.368861
2020-07-06 20:01:30,367 train INFO: validate at epoch 66, top1=88.26000, top5=99.46000, loss=0.391440
2020-07-06 20:02:10,249 train INFO: validate at epoch 67, top1=85.64000, top5=99.42000, loss=0.480671
2020-07-06 20:02:48,994 train INFO: validate at epoch 68, top1=85.32000, top5=99.47000, loss=0.520690
2020-07-06 20:03:28,005 train INFO: validate at epoch 69, top1=86.58000, top5=99.56000, loss=0.462732
2020-07-06 20:04:07,148 train INFO: validate at epoch 70, top1=86.05000, top5=99.64000, loss=0.492189
2020-07-06 20:04:46,095 train INFO: validate at epoch 71, top1=88.02000, top5=99.54000, loss=0.403724
2020-07-06 20:05:25,013 train INFO: validate at epoch 72, top1=87.96000, top5=99.52000, loss=0.409592
2020-07-06 20:06:04,268 train INFO: validate at epoch 73, top1=88.81000, top5=99.58000, loss=0.370469
2020-07-06 20:06:39,728 train INFO: validate at epoch 74, top1=88.57000, top5=99.55000, loss=0.395319
2020-07-06 20:07:18,823 train INFO: validate at epoch 75, top1=89.63000, top5=99.55000, loss=0.335497
2020-07-06 20:07:58,112 train INFO: validate at epoch 76, top1=89.10000, top5=99.67000, loss=0.369993
2020-07-06 20:08:37,774 train INFO: validate at epoch 77, top1=87.39000, top5=99.42000, loss=0.453018
2020-07-06 20:09:17,612 train INFO: validate at epoch 78, top1=89.14000, top5=99.57000, loss=0.379931
2020-07-06 20:09:56,612 train INFO: validate at epoch 79, top1=86.52000, top5=99.10000, loss=0.531465
2020-07-06 20:10:35,447 train INFO: validate at epoch 80, top1=87.31000, top5=99.78000, loss=0.438165
2020-07-06 20:11:12,502 train INFO: validate at epoch 81, top1=88.61000, top5=99.50000, loss=0.388805
2020-07-06 20:11:48,400 train INFO: validate at epoch 82, top1=86.49000, top5=99.64000, loss=0.496606
2020-07-06 20:12:27,601 train INFO: validate at epoch 83, top1=88.89000, top5=99.70000, loss=0.391226
2020-07-06 20:13:06,475 train INFO: validate at epoch 84, top1=88.10000, top5=99.53000, loss=0.432252
2020-07-06 20:13:45,571 train INFO: validate at epoch 85, top1=89.71000, top5=99.60000, loss=0.349228
2020-07-06 20:14:23,712 train INFO: validate at epoch 86, top1=87.73000, top5=99.49000, loss=0.448738
2020-07-06 20:15:01,901 train INFO: validate at epoch 87, top1=86.53000, top5=99.52000, loss=0.474809
2020-07-06 20:15:40,716 train INFO: validate at epoch 88, top1=87.04000, top5=99.58000, loss=0.465868
2020-07-06 20:16:16,379 train INFO: validate at epoch 89, top1=87.15000, top5=99.35000, loss=0.438851
2020-07-06 20:16:52,939 train INFO: validate at epoch 90, top1=89.24000, top5=99.62000, loss=0.402890
2020-07-06 20:17:31,535 train INFO: validate at epoch 91, top1=88.08000, top5=99.52000, loss=0.427842
2020-07-06 20:18:10,353 train INFO: validate at epoch 92, top1=89.12000, top5=99.60000, loss=0.368942
2020-07-06 20:18:49,236 train INFO: validate at epoch 93, top1=86.82000, top5=99.66000, loss=0.488902
2020-07-06 20:19:27,982 train INFO: validate at epoch 94, top1=89.20000, top5=99.64000, loss=0.389813
2020-07-06 20:20:07,682 train INFO: validate at epoch 95, top1=89.28000, top5=99.55000, loss=0.369882
2020-07-06 20:20:46,404 train INFO: validate at epoch 96, top1=88.76000, top5=99.49000, loss=0.405588
2020-07-06 20:21:20,543 train INFO: validate at epoch 97, top1=89.77000, top5=99.67000, loss=0.377457
2020-07-06 20:21:59,135 train INFO: validate at epoch 98, top1=88.86000, top5=99.78000, loss=0.397801
2020-07-06 20:22:39,607 train INFO: validate at epoch 99, top1=88.34000, top5=99.51000, loss=0.398535
2020-07-06 20:23:18,851 train INFO: validate at epoch 100, top1=89.55000, top5=99.62000, loss=0.355268
2020-07-06 20:23:58,419 train INFO: validate at epoch 101, top1=89.28000, top5=99.56000, loss=0.363890
2020-07-06 20:24:36,926 train INFO: validate at epoch 102, top1=88.66000, top5=99.38000, loss=0.411958
2020-07-06 20:25:14,894 train INFO: validate at epoch 103, top1=86.87000, top5=99.50000, loss=0.500681
2020-07-06 20:25:52,995 train INFO: validate at epoch 104, top1=88.75000, top5=99.68000, loss=0.389094
2020-07-06 20:26:26,028 train INFO: validate at epoch 105, top1=89.28000, top5=99.56000, loss=0.384883
2020-07-06 20:27:04,607 train INFO: validate at epoch 106, top1=89.20000, top5=99.52000, loss=0.379228
2020-07-06 20:27:43,621 train INFO: validate at epoch 107, top1=89.16000, top5=99.61000, loss=0.398625
2020-07-06 20:28:22,191 train INFO: validate at epoch 108, top1=88.02000, top5=99.24000, loss=0.447359
2020-07-06 20:29:00,993 train INFO: validate at epoch 109, top1=88.50000, top5=99.46000, loss=0.405393
2020-07-06 20:29:40,034 train INFO: validate at epoch 110, top1=89.50000, top5=99.68000, loss=0.382862
2020-07-06 20:30:19,189 train INFO: validate at epoch 111, top1=88.96000, top5=99.53000, loss=0.412127
2020-07-06 20:30:58,349 train INFO: validate at epoch 112, top1=87.03000, top5=99.61000, loss=0.486501
2020-07-06 20:31:34,113 train INFO: validate at epoch 113, top1=89.47000, top5=99.58000, loss=0.370806
2020-07-06 20:32:13,304 train INFO: validate at epoch 114, top1=89.84000, top5=99.66000, loss=0.349207
2020-07-06 20:32:52,037 train INFO: validate at epoch 115, top1=89.92000, top5=99.41000, loss=0.355605
2020-07-06 20:33:31,247 train INFO: validate at epoch 116, top1=87.92000, top5=99.45000, loss=0.443881
2020-07-06 20:34:09,675 train INFO: validate at epoch 117, top1=89.41000, top5=99.55000, loss=0.392124
2020-07-06 20:34:48,437 train INFO: validate at epoch 118, top1=89.63000, top5=99.58000, loss=0.391347
2020-07-06 20:35:27,383 train INFO: validate at epoch 119, top1=89.24000, top5=99.63000, loss=0.376553
2020-07-06 20:36:06,132 train INFO: validate at epoch 120, top1=88.25000, top5=99.48000, loss=0.460242
2020-07-06 20:36:42,057 train INFO: validate at epoch 121, top1=89.59000, top5=99.64000, loss=0.373734
2020-07-06 20:37:21,193 train INFO: validate at epoch 122, top1=89.57000, top5=99.67000, loss=0.381420
2020-07-06 20:37:59,888 train INFO: validate at epoch 123, top1=89.53000, top5=99.51000, loss=0.394339
2020-07-06 20:38:38,931 train INFO: validate at epoch 124, top1=89.40000, top5=99.62000, loss=0.385081
2020-07-06 20:39:17,948 train INFO: validate at epoch 125, top1=88.79000, top5=99.65000, loss=0.377545
2020-07-06 20:39:56,911 train INFO: validate at epoch 126, top1=89.89000, top5=99.60000, loss=0.356033
2020-07-06 20:40:35,774 train INFO: validate at epoch 127, top1=87.76000, top5=99.33000, loss=0.447687
2020-07-06 20:41:13,117 train INFO: validate at epoch 128, top1=89.86000, top5=99.62000, loss=0.367006
2020-07-06 20:41:51,827 train INFO: validate at epoch 129, top1=88.61000, top5=99.57000, loss=0.416805
2020-07-06 20:42:30,996 train INFO: validate at epoch 130, top1=88.36000, top5=99.44000, loss=0.443189
2020-07-06 20:43:10,249 train INFO: validate at epoch 131, top1=90.02000, top5=99.72000, loss=0.352311
2020-07-06 20:43:49,899 train INFO: validate at epoch 132, top1=89.85000, top5=99.67000, loss=0.377602
2020-07-06 20:44:29,253 train INFO: validate at epoch 133, top1=89.50000, top5=99.69000, loss=0.360660
2020-07-06 20:45:05,028 train INFO: validate at epoch 134, top1=90.17000, top5=99.60000, loss=0.357067
2020-07-06 20:45:35,895 train INFO: validate at epoch 135, top1=88.91000, top5=99.59000, loss=0.400647
2020-07-06 20:46:06,822 train INFO: validate at epoch 136, top1=89.61000, top5=99.58000, loss=0.378419
2020-07-06 20:46:37,186 train INFO: validate at epoch 137, top1=90.64000, top5=99.58000, loss=0.347043
2020-07-06 20:47:07,715 train INFO: validate at epoch 138, top1=89.33000, top5=99.56000, loss=0.398530
2020-07-06 20:47:39,073 train INFO: validate at epoch 139, top1=88.54000, top5=99.63000, loss=0.430177
2020-07-06 20:48:09,119 train INFO: validate at epoch 140, top1=90.72000, top5=99.61000, loss=0.332399
2020-07-06 20:48:39,799 train INFO: validate at epoch 141, top1=89.94000, top5=99.53000, loss=0.356657
2020-07-06 20:49:10,498 train INFO: validate at epoch 142, top1=87.11000, top5=99.24000, loss=0.529899
2020-07-06 20:49:40,755 train INFO: validate at epoch 143, top1=89.79000, top5=99.53000, loss=0.385842
2020-07-06 20:50:11,144 train INFO: validate at epoch 144, top1=89.53000, top5=99.61000, loss=0.388235
2020-07-06 20:50:42,071 train INFO: validate at epoch 145, top1=88.63000, top5=99.51000, loss=0.441688
2020-07-06 20:51:13,555 train INFO: validate at epoch 146, top1=89.00000, top5=99.62000, loss=0.390710
2020-07-06 20:51:45,339 train INFO: validate at epoch 147, top1=90.51000, top5=99.52000, loss=0.361326
2020-07-06 20:52:17,057 train INFO: validate at epoch 148, top1=90.11000, top5=99.57000, loss=0.379222
2020-07-06 20:52:48,195 train INFO: validate at epoch 149, top1=86.47000, top5=99.45000, loss=0.553987
2020-07-06 20:53:18,774 train INFO: validate at epoch 150, top1=89.90000, top5=99.66000, loss=0.347558
2020-07-06 20:53:49,576 train INFO: validate at epoch 151, top1=89.68000, top5=99.69000, loss=0.364140
2020-07-06 20:54:20,935 train INFO: validate at epoch 152, top1=89.92000, top5=99.63000, loss=0.377730
2020-07-06 20:54:52,055 train INFO: validate at epoch 153, top1=90.79000, top5=99.71000, loss=0.333121
2020-07-06 20:55:22,792 train INFO: validate at epoch 154, top1=89.57000, top5=99.67000, loss=0.390279
2020-07-06 20:55:53,741 train INFO: validate at epoch 155, top1=89.71000, top5=99.70000, loss=0.384872
2020-07-06 20:56:25,626 train INFO: validate at epoch 156, top1=88.34000, top5=99.51000, loss=0.431583
2020-07-06 20:57:04,527 train INFO: validate at epoch 157, top1=87.76000, top5=99.57000, loss=0.470467
2020-07-06 20:57:44,080 train INFO: validate at epoch 158, top1=88.38000, top5=99.58000, loss=0.438224
2020-07-06 20:58:23,131 train INFO: validate at epoch 159, top1=89.84000, top5=99.66000, loss=0.377252
2020-07-06 20:59:03,129 train INFO: validate at epoch 160, top1=90.20000, top5=99.65000, loss=0.355539
2020-07-06 20:59:42,599 train INFO: validate at epoch 161, top1=90.52000, top5=99.64000, loss=0.363454
2020-07-06 21:00:22,390 train INFO: validate at epoch 162, top1=89.59000, top5=99.52000, loss=0.388141
2020-07-06 21:01:02,032 train INFO: validate at epoch 163, top1=89.62000, top5=99.57000, loss=0.387200
2020-07-06 21:01:40,664 train INFO: validate at epoch 164, top1=88.17000, top5=99.57000, loss=0.477819
2020-07-06 21:02:20,095 train INFO: validate at epoch 165, top1=89.48000, top5=99.61000, loss=0.378500
2020-07-06 21:02:59,510 train INFO: validate at epoch 166, top1=90.49000, top5=99.58000, loss=0.344320
2020-07-06 21:03:39,052 train INFO: validate at epoch 167, top1=88.93000, top5=99.59000, loss=0.401268
2020-07-06 21:04:18,794 train INFO: validate at epoch 168, top1=90.35000, top5=99.76000, loss=0.364570
2020-07-06 21:04:58,112 train INFO: validate at epoch 169, top1=90.02000, top5=99.52000, loss=0.386227
2020-07-06 21:05:37,443 train INFO: validate at epoch 170, top1=88.60000, top5=99.39000, loss=0.442630
2020-07-06 21:06:17,121 train INFO: validate at epoch 171, top1=89.58000, top5=99.55000, loss=0.376856
2020-07-06 21:06:55,536 train INFO: validate at epoch 172, top1=88.40000, top5=99.66000, loss=0.425256
2020-07-06 21:07:34,980 train INFO: validate at epoch 173, top1=90.23000, top5=99.62000, loss=0.367300
2020-07-06 21:08:14,597 train INFO: validate at epoch 174, top1=88.68000, top5=99.41000, loss=0.439452
2020-07-06 21:08:54,121 train INFO: validate at epoch 175, top1=90.37000, top5=99.67000, loss=0.338244
2020-07-06 21:09:33,720 train INFO: validate at epoch 176, top1=88.64000, top5=99.61000, loss=0.436530
2020-07-06 21:10:13,062 train INFO: validate at epoch 177, top1=90.33000, top5=99.62000, loss=0.384300
2020-07-06 21:10:52,322 train INFO: validate at epoch 178, top1=91.12000, top5=99.62000, loss=0.341465
2020-07-06 21:11:30,395 train INFO: validate at epoch 179, top1=89.71000, top5=99.48000, loss=0.380116
2020-07-06 21:12:09,478 train INFO: validate at epoch 180, top1=89.65000, top5=99.56000, loss=0.381436
2020-07-06 21:12:48,420 train INFO: validate at epoch 181, top1=90.68000, top5=99.65000, loss=0.347007
2020-07-06 21:13:27,476 train INFO: validate at epoch 182, top1=89.62000, top5=99.53000, loss=0.385004
2020-07-06 21:14:06,928 train INFO: validate at epoch 183, top1=87.03000, top5=99.14000, loss=0.491865
2020-07-06 21:14:46,089 train INFO: validate at epoch 184, top1=90.44000, top5=99.60000, loss=0.349698
2020-07-06 21:15:25,274 train INFO: validate at epoch 185, top1=89.89000, top5=99.66000, loss=0.364372
2020-07-06 21:16:04,486 train INFO: validate at epoch 186, top1=89.57000, top5=99.69000, loss=0.407799
2020-07-06 21:16:41,499 train INFO: validate at epoch 187, top1=89.84000, top5=99.64000, loss=0.371321
2020-07-06 21:17:20,662 train INFO: validate at epoch 188, top1=89.98000, top5=99.63000, loss=0.374308
2020-07-06 21:17:59,960 train INFO: validate at epoch 189, top1=90.29000, top5=99.65000, loss=0.362521
2020-07-06 21:18:38,985 train INFO: validate at epoch 190, top1=90.59000, top5=99.60000, loss=0.364863
2020-07-06 21:19:17,860 train INFO: validate at epoch 191, top1=90.60000, top5=99.66000, loss=0.339323
2020-07-06 21:19:57,179 train INFO: validate at epoch 192, top1=90.01000, top5=99.58000, loss=0.356536
2020-07-06 21:20:36,074 train INFO: validate at epoch 193, top1=89.95000, top5=99.58000, loss=0.375739
2020-07-06 21:21:15,045 train INFO: validate at epoch 194, top1=88.78000, top5=99.54000, loss=0.414765
2020-07-06 21:21:51,605 train INFO: validate at epoch 195, top1=89.13000, top5=99.39000, loss=0.429243
2020-07-06 21:22:30,313 train INFO: validate at epoch 196, top1=89.63000, top5=99.67000, loss=0.374714
2020-07-06 21:23:09,612 train INFO: validate at epoch 197, top1=89.16000, top5=99.44000, loss=0.423222
2020-07-06 21:23:48,740 train INFO: validate at epoch 198, top1=89.51000, top5=99.61000, loss=0.410378
2020-07-06 21:24:27,514 train INFO: validate at epoch 199, top1=90.79000, top5=99.68000, loss=0.333328
2020-07-06 21:25:06,520 train INFO: validate at epoch 200, top1=90.35000, top5=99.66000, loss=0.364336
2020-07-06 21:25:45,994 train INFO: validate at epoch 201, top1=92.78000, top5=99.84000, loss=0.254394
2020-07-06 21:26:23,846 train INFO: validate at epoch 202, top1=93.05000, top5=99.86000, loss=0.255252
2020-07-06 21:27:03,242 train INFO: validate at epoch 203, top1=93.16000, top5=99.83000, loss=0.257182
2020-07-06 21:27:42,763 train INFO: validate at epoch 204, top1=93.24000, top5=99.84000, loss=0.254531
2020-07-06 21:28:22,142 train INFO: validate at epoch 205, top1=93.33000, top5=99.85000, loss=0.261087
2020-07-06 21:29:01,901 train INFO: validate at epoch 206, top1=93.44000, top5=99.83000, loss=0.264533
2020-07-06 21:29:41,302 train INFO: validate at epoch 207, top1=93.52000, top5=99.87000, loss=0.264283
2020-07-06 21:30:20,705 train INFO: validate at epoch 208, top1=93.50000, top5=99.86000, loss=0.268561
2020-07-06 21:31:00,185 train INFO: validate at epoch 209, top1=93.60000, top5=99.81000, loss=0.266060
2020-07-06 21:31:38,340 train INFO: validate at epoch 210, top1=93.60000, top5=99.82000, loss=0.270895
2020-07-06 21:32:17,259 train INFO: validate at epoch 211, top1=93.70000, top5=99.83000, loss=0.270259
2020-07-06 21:32:56,622 train INFO: validate at epoch 212, top1=93.64000, top5=99.82000, loss=0.277377
2020-07-06 21:33:35,720 train INFO: validate at epoch 213, top1=93.62000, top5=99.84000, loss=0.280970
2020-07-06 21:34:14,736 train INFO: validate at epoch 214, top1=93.67000, top5=99.85000, loss=0.282947
2020-07-06 21:34:53,632 train INFO: validate at epoch 215, top1=93.77000, top5=99.85000, loss=0.285512
2020-07-06 21:35:32,538 train INFO: validate at epoch 216, top1=93.74000, top5=99.86000, loss=0.285152
2020-07-06 21:36:11,509 train INFO: validate at epoch 217, top1=93.76000, top5=99.85000, loss=0.289063
2020-07-06 21:36:49,124 train INFO: validate at epoch 218, top1=93.60000, top5=99.86000, loss=0.296104
2020-07-06 21:37:28,002 train INFO: validate at epoch 219, top1=93.61000, top5=99.84000, loss=0.296387
2020-07-06 21:38:06,747 train INFO: validate at epoch 220, top1=93.64000, top5=99.85000, loss=0.295399
2020-07-06 21:38:45,996 train INFO: validate at epoch 221, top1=93.60000, top5=99.86000, loss=0.296143
2020-07-06 21:39:25,076 train INFO: validate at epoch 222, top1=93.76000, top5=99.84000, loss=0.291981
2020-07-06 21:40:03,967 train INFO: validate at epoch 223, top1=93.76000, top5=99.82000, loss=0.290690
2020-07-06 21:40:43,463 train INFO: validate at epoch 224, top1=93.61000, top5=99.85000, loss=0.298818
2020-07-06 21:41:21,748 train INFO: validate at epoch 225, top1=93.65000, top5=99.81000, loss=0.303589
2020-07-06 21:42:00,488 train INFO: validate at epoch 226, top1=93.70000, top5=99.82000, loss=0.301618
2020-07-06 21:42:39,368 train INFO: validate at epoch 227, top1=93.73000, top5=99.86000, loss=0.300731
2020-07-06 21:43:18,026 train INFO: validate at epoch 228, top1=93.63000, top5=99.81000, loss=0.305594
2020-07-06 21:43:57,114 train INFO: validate at epoch 229, top1=93.63000, top5=99.84000, loss=0.304876
2020-07-06 21:44:36,163 train INFO: validate at epoch 230, top1=93.66000, top5=99.83000, loss=0.304976
2020-07-06 21:45:15,943 train INFO: validate at epoch 231, top1=93.72000, top5=99.81000, loss=0.306050
2020-07-06 21:45:54,894 train INFO: validate at epoch 232, top1=93.68000, top5=99.81000, loss=0.308473
2020-07-06 21:46:31,808 train INFO: validate at epoch 233, top1=93.85000, top5=99.80000, loss=0.311534
2020-07-06 21:47:10,643 train INFO: validate at epoch 234, top1=93.68000, top5=99.79000, loss=0.311679
2020-07-06 21:47:49,977 train INFO: validate at epoch 235, top1=93.63000, top5=99.82000, loss=0.316147
2020-07-06 21:48:29,520 train INFO: validate at epoch 236, top1=93.81000, top5=99.79000, loss=0.311823
2020-07-06 21:49:08,541 train INFO: validate at epoch 237, top1=93.69000, top5=99.84000, loss=0.314528
2020-07-06 21:49:47,758 train INFO: validate at epoch 238, top1=93.70000, top5=99.82000, loss=0.315550
2020-07-06 21:50:27,246 train INFO: validate at epoch 239, top1=93.72000, top5=99.86000, loss=0.312367
2020-07-06 21:51:06,176 train INFO: validate at epoch 240, top1=93.77000, top5=99.85000, loss=0.317832
2020-07-06 21:51:42,698 train INFO: validate at epoch 241, top1=93.68000, top5=99.84000, loss=0.314477
2020-07-06 21:52:22,046 train INFO: validate at epoch 242, top1=93.87000, top5=99.82000, loss=0.315709
2020-07-06 21:53:00,807 train INFO: validate at epoch 243, top1=93.84000, top5=99.82000, loss=0.321661
2020-07-06 21:53:39,607 train INFO: validate at epoch 244, top1=93.77000, top5=99.85000, loss=0.317939
2020-07-06 21:54:18,239 train INFO: validate at epoch 245, top1=93.80000, top5=99.83000, loss=0.319674
2020-07-06 21:54:56,888 train INFO: validate at epoch 246, top1=93.66000, top5=99.80000, loss=0.313259
2020-07-06 21:55:35,536 train INFO: validate at epoch 247, top1=93.78000, top5=99.79000, loss=0.318114
2020-07-06 21:56:14,682 train INFO: validate at epoch 248, top1=93.82000, top5=99.79000, loss=0.320576
2020-07-06 21:56:51,248 train INFO: validate at epoch 249, top1=93.88000, top5=99.80000, loss=0.323096
2020-07-06 21:57:30,438 train INFO: validate at epoch 250, top1=93.62000, top5=99.81000, loss=0.325390
2020-07-06 21:58:09,661 train INFO: validate at epoch 251, top1=93.86000, top5=99.78000, loss=0.322690
2020-07-06 21:58:47,862 train INFO: validate at epoch 252, top1=93.69000, top5=99.77000, loss=0.326723
2020-07-06 21:59:26,803 train INFO: validate at epoch 253, top1=93.83000, top5=99.80000, loss=0.329663
2020-07-06 22:00:04,775 train INFO: validate at epoch 254, top1=93.83000, top5=99.79000, loss=0.323741
2020-07-06 22:00:43,540 train INFO: validate at epoch 255, top1=93.87000, top5=99.80000, loss=0.323886
2020-07-06 22:01:21,513 train INFO: validate at epoch 256, top1=93.76000, top5=99.82000, loss=0.327421
2020-07-06 22:01:59,180 train INFO: validate at epoch 257, top1=93.67000, top5=99.80000, loss=0.331439
2020-07-06 22:02:37,325 train INFO: validate at epoch 258, top1=93.69000, top5=99.81000, loss=0.331318
2020-07-06 22:03:16,452 train INFO: validate at epoch 259, top1=93.63000, top5=99.82000, loss=0.329237
2020-07-06 22:03:54,133 train INFO: validate at epoch 260, top1=93.65000, top5=99.80000, loss=0.330478
2020-07-06 22:04:32,289 train INFO: validate at epoch 261, top1=93.67000, top5=99.78000, loss=0.332581
2020-07-06 22:05:11,388 train INFO: validate at epoch 262, top1=93.70000, top5=99.81000, loss=0.330942
2020-07-06 22:05:50,078 train INFO: validate at epoch 263, top1=93.76000, top5=99.81000, loss=0.330186
2020-07-06 22:06:27,240 train INFO: validate at epoch 264, top1=93.85000, top5=99.82000, loss=0.333057
2020-07-06 22:07:06,286 train INFO: validate at epoch 265, top1=93.86000, top5=99.81000, loss=0.332348
2020-07-06 22:07:44,556 train INFO: validate at epoch 266, top1=93.83000, top5=99.80000, loss=0.335007
2020-07-06 22:08:23,479 train INFO: validate at epoch 267, top1=93.88000, top5=99.83000, loss=0.331566
2020-07-06 22:09:02,322 train INFO: validate at epoch 268, top1=93.86000, top5=99.81000, loss=0.333488
2020-07-06 22:09:41,057 train INFO: validate at epoch 269, top1=93.81000, top5=99.80000, loss=0.337440
2020-07-06 22:10:19,309 train INFO: validate at epoch 270, top1=93.80000, top5=99.78000, loss=0.339877
2020-07-06 22:10:58,331 train INFO: validate at epoch 271, top1=93.89000, top5=99.80000, loss=0.333599
2020-07-06 22:11:35,008 train INFO: validate at epoch 272, top1=93.77000, top5=99.83000, loss=0.338337
2020-07-06 22:12:13,545 train INFO: validate at epoch 273, top1=93.79000, top5=99.82000, loss=0.336797
2020-07-06 22:12:52,404 train INFO: validate at epoch 274, top1=93.89000, top5=99.82000, loss=0.337562
2020-07-06 22:13:30,866 train INFO: validate at epoch 275, top1=93.90000, top5=99.79000, loss=0.339735
2020-07-06 22:14:08,613 train INFO: validate at epoch 276, top1=93.84000, top5=99.81000, loss=0.346171
2020-07-06 22:14:47,393 train INFO: validate at epoch 277, top1=93.81000, top5=99.84000, loss=0.341056
2020-07-06 22:15:26,040 train INFO: validate at epoch 278, top1=93.87000, top5=99.81000, loss=0.341158
2020-07-06 22:16:04,601 train INFO: validate at epoch 279, top1=93.74000, top5=99.81000, loss=0.342787
2020-07-06 22:16:40,937 train INFO: validate at epoch 280, top1=93.80000, top5=99.83000, loss=0.338929
2020-07-06 22:17:19,762 train INFO: validate at epoch 281, top1=94.05000, top5=99.80000, loss=0.340685
2020-07-06 22:17:58,723 train INFO: validate at epoch 282, top1=93.90000, top5=99.81000, loss=0.336521
2020-07-06 22:18:37,245 train INFO: validate at epoch 283, top1=93.96000, top5=99.83000, loss=0.336726
2020-07-06 22:19:16,012 train INFO: validate at epoch 284, top1=93.94000, top5=99.86000, loss=0.332452
2020-07-06 22:19:53,598 train INFO: validate at epoch 285, top1=93.92000, top5=99.83000, loss=0.341481
2020-07-06 22:20:32,992 train INFO: validate at epoch 286, top1=93.87000, top5=99.79000, loss=0.333922
2020-07-06 22:21:11,639 train INFO: validate at epoch 287, top1=94.07000, top5=99.77000, loss=0.333662
2020-07-06 22:21:47,227 train INFO: validate at epoch 288, top1=93.85000, top5=99.76000, loss=0.343069
2020-07-06 22:22:26,398 train INFO: validate at epoch 289, top1=94.01000, top5=99.80000, loss=0.332719
2020-07-06 22:23:04,918 train INFO: validate at epoch 290, top1=93.76000, top5=99.78000, loss=0.338137
2020-07-06 22:23:43,657 train INFO: validate at epoch 291, top1=93.99000, top5=99.82000, loss=0.338300
2020-07-06 22:24:22,576 train INFO: validate at epoch 292, top1=93.80000, top5=99.80000, loss=0.337551
2020-07-06 22:25:01,587 train INFO: validate at epoch 293, top1=94.05000, top5=99.80000, loss=0.336416
2020-07-06 22:25:40,265 train INFO: validate at epoch 294, top1=93.95000, top5=99.78000, loss=0.336512
2020-07-06 22:26:19,076 train INFO: validate at epoch 295, top1=93.87000, top5=99.83000, loss=0.337642
2020-07-06 22:26:54,994 train INFO: validate at epoch 296, top1=93.97000, top5=99.79000, loss=0.335433
2020-07-06 22:27:33,777 train INFO: validate at epoch 297, top1=93.87000, top5=99.81000, loss=0.341969
2020-07-06 22:28:12,827 train INFO: validate at epoch 298, top1=93.92000, top5=99.80000, loss=0.340940
2020-07-06 22:28:52,039 train INFO: validate at epoch 299, top1=94.09000, top5=99.82000, loss=0.340936
2020-07-06 22:29:31,234 train INFO: validate at epoch 300, top1=93.85000, top5=99.79000, loss=0.334492
2020-07-06 22:30:10,244 train INFO: validate at epoch 301, top1=93.85000, top5=99.78000, loss=0.338587
2020-07-06 22:30:49,283 train INFO: validate at epoch 302, top1=93.85000, top5=99.82000, loss=0.341136
2020-07-06 22:31:25,940 train INFO: validate at epoch 303, top1=93.88000, top5=99.82000, loss=0.342856
2020-07-06 22:32:03,510 train INFO: validate at epoch 304, top1=93.80000, top5=99.79000, loss=0.342977
2020-07-06 22:32:42,643 train INFO: validate at epoch 305, top1=93.78000, top5=99.78000, loss=0.344190
2020-07-06 22:33:21,377 train INFO: validate at epoch 306, top1=93.67000, top5=99.78000, loss=0.346045
2020-07-06 22:34:01,126 train INFO: validate at epoch 307, top1=93.75000, top5=99.79000, loss=0.349286
2020-07-06 22:34:40,059 train INFO: validate at epoch 308, top1=93.71000, top5=99.79000, loss=0.347983
2020-07-06 22:35:18,835 train INFO: validate at epoch 309, top1=93.87000, top5=99.75000, loss=0.341086
2020-07-06 22:35:58,077 train INFO: validate at epoch 310, top1=93.91000, top5=99.79000, loss=0.344913
2020-07-06 22:36:32,583 train INFO: validate at epoch 311, top1=94.02000, top5=99.76000, loss=0.341119
2020-07-06 22:37:11,143 train INFO: validate at epoch 312, top1=94.09000, top5=99.76000, loss=0.344007
2020-07-06 22:37:50,036 train INFO: validate at epoch 313, top1=94.04000, top5=99.76000, loss=0.343242
2020-07-06 22:38:29,791 train INFO: validate at epoch 314, top1=93.70000, top5=99.77000, loss=0.348200
2020-07-06 22:39:08,103 train INFO: validate at epoch 315, top1=93.89000, top5=99.81000, loss=0.348597
2020-07-06 22:39:46,915 train INFO: validate at epoch 316, top1=93.87000, top5=99.77000, loss=0.348513
2020-07-06 22:40:25,887 train INFO: validate at epoch 317, top1=93.66000, top5=99.76000, loss=0.351557
2020-07-06 22:41:04,515 train INFO: validate at epoch 318, top1=93.73000, top5=99.72000, loss=0.353629
2020-07-06 22:41:40,010 train INFO: validate at epoch 319, top1=93.78000, top5=99.73000, loss=0.347614
2020-07-06 22:42:19,541 train INFO: validate at epoch 320, top1=93.63000, top5=99.79000, loss=0.349195
2020-07-06 22:42:58,644 train INFO: validate at epoch 321, top1=93.76000, top5=99.73000, loss=0.347784
2020-07-06 22:43:37,810 train INFO: validate at epoch 322, top1=93.73000, top5=99.74000, loss=0.354963
2020-07-06 22:44:16,515 train INFO: validate at epoch 323, top1=93.81000, top5=99.74000, loss=0.348684
2020-07-06 22:44:55,494 train INFO: validate at epoch 324, top1=93.88000, top5=99.81000, loss=0.339162
2020-07-06 22:45:35,224 train INFO: validate at epoch 325, top1=93.81000, top5=99.78000, loss=0.340504
2020-07-06 22:46:14,643 train INFO: validate at epoch 326, top1=93.90000, top5=99.76000, loss=0.333784
2020-07-06 22:46:50,185 train INFO: validate at epoch 327, top1=93.84000, top5=99.74000, loss=0.345181
2020-07-06 22:47:29,773 train INFO: validate at epoch 328, top1=93.88000, top5=99.78000, loss=0.345100
2020-07-06 22:48:08,883 train INFO: validate at epoch 329, top1=93.76000, top5=99.77000, loss=0.346561
2020-07-06 22:48:48,381 train INFO: validate at epoch 330, top1=93.62000, top5=99.79000, loss=0.350330
2020-07-06 22:49:28,209 train INFO: validate at epoch 331, top1=93.65000, top5=99.75000, loss=0.347678
2020-07-06 22:50:07,531 train INFO: validate at epoch 332, top1=93.91000, top5=99.74000, loss=0.348094
2020-07-06 22:50:46,969 train INFO: validate at epoch 333, top1=93.81000, top5=99.77000, loss=0.338875
2020-07-06 22:51:25,076 train INFO: validate at epoch 334, top1=93.61000, top5=99.73000, loss=0.346629
2020-07-06 22:52:04,337 train INFO: validate at epoch 335, top1=93.81000, top5=99.70000, loss=0.341897
2020-07-06 22:52:43,706 train INFO: validate at epoch 336, top1=93.88000, top5=99.77000, loss=0.339951
2020-07-06 22:53:22,798 train INFO: validate at epoch 337, top1=93.85000, top5=99.77000, loss=0.348255
2020-07-06 22:54:01,444 train INFO: validate at epoch 338, top1=93.77000, top5=99.74000, loss=0.347673
2020-07-06 22:54:41,022 train INFO: validate at epoch 339, top1=93.68000, top5=99.73000, loss=0.343575
2020-07-06 22:55:21,108 train INFO: validate at epoch 340, top1=93.90000, top5=99.73000, loss=0.343065
2020-07-06 22:56:00,147 train INFO: validate at epoch 341, top1=93.84000, top5=99.72000, loss=0.350569
2020-07-06 22:56:38,352 train INFO: validate at epoch 342, top1=93.89000, top5=99.69000, loss=0.342148
2020-07-06 22:57:17,834 train INFO: validate at epoch 343, top1=93.86000, top5=99.71000, loss=0.339955
2020-07-06 22:57:56,719 train INFO: validate at epoch 344, top1=93.93000, top5=99.68000, loss=0.346305
2020-07-06 22:58:35,670 train INFO: validate at epoch 345, top1=93.90000, top5=99.77000, loss=0.350702
2020-07-06 22:59:14,885 train INFO: validate at epoch 346, top1=94.06000, top5=99.69000, loss=0.349506
2020-07-06 22:59:53,746 train INFO: validate at epoch 347, top1=93.84000, top5=99.71000, loss=0.350528
2020-07-06 23:00:33,245 train INFO: validate at epoch 348, top1=93.84000, top5=99.67000, loss=0.358290
2020-07-06 23:01:12,353 train INFO: validate at epoch 349, top1=93.81000, top5=99.69000, loss=0.347266
2020-07-06 23:01:50,021 train INFO: validate at epoch 350, top1=93.81000, top5=99.75000, loss=0.343894
2020-07-06 23:02:29,536 train INFO: validate at epoch 351, top1=93.77000, top5=99.73000, loss=0.346482
2020-07-06 23:03:08,597 train INFO: validate at epoch 352, top1=93.69000, top5=99.73000, loss=0.348994
2020-07-06 23:03:47,922 train INFO: validate at epoch 353, top1=93.67000, top5=99.79000, loss=0.359570
2020-07-06 23:04:26,956 train INFO: validate at epoch 354, top1=93.87000, top5=99.75000, loss=0.361510
2020-07-06 23:05:06,211 train INFO: validate at epoch 355, top1=93.79000, top5=99.71000, loss=0.358961
2020-07-06 23:05:45,191 train INFO: validate at epoch 356, top1=93.55000, top5=99.74000, loss=0.367053
2020-07-06 23:06:23,425 train INFO: validate at epoch 357, top1=93.55000, top5=99.74000, loss=0.369611
2020-07-06 23:07:00,911 train INFO: validate at epoch 358, top1=93.65000, top5=99.65000, loss=0.355879
2020-07-06 23:07:40,140 train INFO: validate at epoch 359, top1=93.51000, top5=99.69000, loss=0.365409
2020-07-06 23:08:18,980 train INFO: validate at epoch 360, top1=93.68000, top5=99.73000, loss=0.359785
2020-07-06 23:08:57,494 train INFO: validate at epoch 361, top1=93.74000, top5=99.79000, loss=0.356470
2020-07-06 23:09:36,761 train INFO: validate at epoch 362, top1=93.48000, top5=99.76000, loss=0.367097
2020-07-06 23:10:15,148 train INFO: validate at epoch 363, top1=93.38000, top5=99.72000, loss=0.363374
2020-07-06 23:10:54,083 train INFO: validate at epoch 364, top1=93.47000, top5=99.69000, loss=0.361141
2020-07-06 23:11:30,213 train INFO: validate at epoch 365, top1=93.76000, top5=99.71000, loss=0.354626
2020-07-06 23:12:09,146 train INFO: validate at epoch 366, top1=93.63000, top5=99.71000, loss=0.358323
2020-07-06 23:12:48,512 train INFO: validate at epoch 367, top1=93.63000, top5=99.75000, loss=0.359310
2020-07-06 23:13:27,301 train INFO: validate at epoch 368, top1=93.61000, top5=99.75000, loss=0.359930
2020-07-06 23:14:06,259 train INFO: validate at epoch 369, top1=93.48000, top5=99.69000, loss=0.365416
2020-07-06 23:14:45,200 train INFO: validate at epoch 370, top1=93.40000, top5=99.72000, loss=0.364726
2020-07-06 23:15:23,731 train INFO: validate at epoch 371, top1=93.17000, top5=99.73000, loss=0.373655
2020-07-06 23:16:02,287 train INFO: validate at epoch 372, top1=93.46000, top5=99.75000, loss=0.365434
2020-07-06 23:16:36,935 train INFO: validate at epoch 373, top1=93.66000, top5=99.69000, loss=0.367591
2020-07-06 23:17:15,749 train INFO: validate at epoch 374, top1=93.50000, top5=99.69000, loss=0.372491
2020-07-06 23:17:54,570 train INFO: validate at epoch 375, top1=93.45000, top5=99.70000, loss=0.373642
2020-07-06 23:18:33,922 train INFO: validate at epoch 376, top1=93.44000, top5=99.75000, loss=0.369372
2020-07-06 23:19:12,600 train INFO: validate at epoch 377, top1=93.39000, top5=99.70000, loss=0.372036
2020-07-06 23:19:51,674 train INFO: validate at epoch 378, top1=93.41000, top5=99.72000, loss=0.373020
2020-07-06 23:20:30,607 train INFO: validate at epoch 379, top1=93.14000, top5=99.71000, loss=0.372536
2020-07-06 23:21:09,275 train INFO: validate at epoch 380, top1=93.48000, top5=99.72000, loss=0.357327
2020-07-06 23:21:45,371 train INFO: validate at epoch 381, top1=93.39000, top5=99.73000, loss=0.359021
2020-07-06 23:22:25,392 train INFO: validate at epoch 382, top1=93.63000, top5=99.74000, loss=0.354145
2020-07-06 23:23:04,029 train INFO: validate at epoch 383, top1=92.95000, top5=99.73000, loss=0.398351
2020-07-06 23:23:43,131 train INFO: validate at epoch 384, top1=93.21000, top5=99.77000, loss=0.374684
2020-07-06 23:24:22,488 train INFO: validate at epoch 385, top1=93.36000, top5=99.74000, loss=0.377839
2020-07-06 23:25:01,307 train INFO: validate at epoch 386, top1=93.53000, top5=99.71000, loss=0.368402
2020-07-06 23:25:40,613 train INFO: validate at epoch 387, top1=93.29000, top5=99.74000, loss=0.374387
2020-07-06 23:26:19,252 train INFO: validate at epoch 388, top1=93.48000, top5=99.79000, loss=0.356092
2020-07-06 23:26:55,572 train INFO: validate at epoch 389, top1=93.49000, top5=99.67000, loss=0.362736
2020-07-06 23:27:34,917 train INFO: validate at epoch 390, top1=93.48000, top5=99.74000, loss=0.345734
2020-07-06 23:28:13,828 train INFO: validate at epoch 391, top1=93.51000, top5=99.72000, loss=0.355306
2020-07-06 23:28:52,543 train INFO: validate at epoch 392, top1=93.29000, top5=99.73000, loss=0.365915
2020-07-06 23:29:31,559 train INFO: validate at epoch 393, top1=93.50000, top5=99.72000, loss=0.343141
2020-07-06 23:30:10,618 train INFO: validate at epoch 394, top1=93.63000, top5=99.78000, loss=0.349400
2020-07-06 23:30:49,364 train INFO: validate at epoch 395, top1=93.26000, top5=99.74000, loss=0.359830
2020-07-06 23:31:27,017 train INFO: validate at epoch 396, top1=93.21000, top5=99.72000, loss=0.363191
2020-07-06 23:32:05,422 train INFO: validate at epoch 397, top1=93.33000, top5=99.73000, loss=0.368592
2020-07-06 23:32:44,044 train INFO: validate at epoch 398, top1=93.15000, top5=99.71000, loss=0.376358
2020-07-06 23:33:23,126 train INFO: validate at epoch 399, top1=93.26000, top5=99.76000, loss=0.368753
2020-07-06 23:34:01,825 train INFO: validate at epoch 400, top1=93.06000, top5=99.74000, loss=0.362050
2020-07-06 23:34:41,508 train INFO: validate at epoch 401, top1=93.42000, top5=99.77000, loss=0.347289
2020-07-06 23:35:20,499 train INFO: validate at epoch 402, top1=93.43000, top5=99.77000, loss=0.343032
2020-07-06 23:35:59,168 train INFO: validate at epoch 403, top1=93.65000, top5=99.78000, loss=0.341038
2020-07-06 23:36:34,953 train INFO: validate at epoch 404, top1=93.61000, top5=99.78000, loss=0.336600
2020-07-06 23:37:13,835 train INFO: validate at epoch 405, top1=93.66000, top5=99.77000, loss=0.340621
2020-07-06 23:37:53,079 train INFO: validate at epoch 406, top1=93.71000, top5=99.73000, loss=0.337045
2020-07-06 23:38:33,115 train INFO: validate at epoch 407, top1=93.62000, top5=99.75000, loss=0.339392
2020-07-06 23:39:11,553 train INFO: validate at epoch 408, top1=93.76000, top5=99.74000, loss=0.336034
2020-07-06 23:39:50,557 train INFO: validate at epoch 409, top1=93.77000, top5=99.79000, loss=0.337129
2020-07-06 23:40:28,257 train INFO: validate at epoch 410, top1=93.74000, top5=99.76000, loss=0.338254
2020-07-06 23:41:06,775 train INFO: validate at epoch 411, top1=93.74000, top5=99.77000, loss=0.339511
2020-07-06 23:41:42,089 train INFO: validate at epoch 412, top1=93.72000, top5=99.78000, loss=0.337076
2020-07-06 23:42:20,556 train INFO: validate at epoch 413, top1=93.80000, top5=99.76000, loss=0.333166
2020-07-06 23:42:59,949 train INFO: validate at epoch 414, top1=93.72000, top5=99.75000, loss=0.339348
2020-07-06 23:43:39,082 train INFO: validate at epoch 415, top1=93.72000, top5=99.75000, loss=0.340663
2020-07-06 23:44:18,065 train INFO: validate at epoch 416, top1=93.77000, top5=99.74000, loss=0.340219
2020-07-06 23:44:56,803 train INFO: validate at epoch 417, top1=93.76000, top5=99.75000, loss=0.338750
2020-07-06 23:45:35,805 train INFO: validate at epoch 418, top1=93.77000, top5=99.78000, loss=0.337406
2020-07-06 23:46:14,822 train INFO: validate at epoch 419, top1=93.76000, top5=99.77000, loss=0.340546
2020-07-06 23:46:51,232 train INFO: validate at epoch 420, top1=93.86000, top5=99.77000, loss=0.342550
2020-07-06 23:47:30,126 train INFO: validate at epoch 421, top1=93.84000, top5=99.80000, loss=0.336636
2020-07-06 23:48:09,290 train INFO: validate at epoch 422, top1=93.75000, top5=99.77000, loss=0.336871
2020-07-06 23:48:47,927 train INFO: validate at epoch 423, top1=93.82000, top5=99.78000, loss=0.335781
2020-07-06 23:49:26,183 train INFO: validate at epoch 424, top1=93.89000, top5=99.73000, loss=0.337070
2020-07-06 23:50:05,437 train INFO: validate at epoch 425, top1=93.84000, top5=99.76000, loss=0.341592
2020-07-06 23:50:44,976 train INFO: validate at epoch 426, top1=93.88000, top5=99.74000, loss=0.340789
2020-07-06 23:51:23,665 train INFO: validate at epoch 427, top1=93.80000, top5=99.74000, loss=0.337828
2020-07-06 23:52:01,343 train INFO: validate at epoch 428, top1=93.89000, top5=99.72000, loss=0.339449
2020-07-06 23:52:40,526 train INFO: validate at epoch 429, top1=93.79000, top5=99.76000, loss=0.338433
2020-07-06 23:53:19,409 train INFO: validate at epoch 430, top1=93.87000, top5=99.76000, loss=0.338675
2020-07-06 23:53:57,472 train INFO: validate at epoch 431, top1=93.87000, top5=99.76000, loss=0.339117
2020-07-06 23:54:35,362 train INFO: validate at epoch 432, top1=93.76000, top5=99.75000, loss=0.341078
2020-07-06 23:55:14,267 train INFO: validate at epoch 433, top1=93.75000, top5=99.75000, loss=0.340157
2020-07-06 23:55:53,217 train INFO: validate at epoch 434, top1=93.83000, top5=99.74000, loss=0.340783
2020-07-06 23:56:30,733 train INFO: validate at epoch 435, top1=93.84000, top5=99.73000, loss=0.341600
2020-07-06 23:57:09,902 train INFO: validate at epoch 436, top1=93.79000, top5=99.74000, loss=0.340398
2020-07-06 23:57:48,716 train INFO: validate at epoch 437, top1=93.88000, top5=99.75000, loss=0.339215
2020-07-06 23:58:28,047 train INFO: validate at epoch 438, top1=93.85000, top5=99.76000, loss=0.341063
2020-07-06 23:59:04,433 train INFO: validate at epoch 439, top1=93.83000, top5=99.77000, loss=0.344524
2020-07-06 23:59:43,819 train INFO: validate at epoch 440, top1=93.84000, top5=99.74000, loss=0.341315
2020-07-07 00:00:23,005 train INFO: validate at epoch 441, top1=93.84000, top5=99.76000, loss=0.342464
2020-07-07 00:01:02,013 train INFO: validate at epoch 442, top1=93.90000, top5=99.73000, loss=0.341156
2020-07-07 00:01:38,328 train INFO: validate at epoch 443, top1=93.93000, top5=99.74000, loss=0.342877
2020-07-07 00:02:17,612 train INFO: validate at epoch 444, top1=93.78000, top5=99.74000, loss=0.342797
2020-07-07 00:02:56,433 train INFO: validate at epoch 445, top1=93.82000, top5=99.73000, loss=0.340518
2020-07-07 00:03:35,236 train INFO: validate at epoch 446, top1=93.86000, top5=99.76000, loss=0.339935
2020-07-07 00:04:13,678 train INFO: validate at epoch 447, top1=93.77000, top5=99.74000, loss=0.337821
2020-07-07 00:04:52,483 train INFO: validate at epoch 448, top1=93.79000, top5=99.74000, loss=0.341622
2020-07-07 00:05:30,591 train INFO: validate at epoch 449, top1=93.89000, top5=99.76000, loss=0.341019
2020-07-07 00:06:08,328 train INFO: validate at epoch 450, top1=93.87000, top5=99.74000, loss=0.339504
2020-07-07 00:06:43,895 train INFO: validate at epoch 451, top1=93.86000, top5=99.73000, loss=0.340394
2020-07-07 00:07:23,044 train INFO: validate at epoch 452, top1=93.91000, top5=99.75000, loss=0.338185
2020-07-07 00:08:02,682 train INFO: validate at epoch 453, top1=93.92000, top5=99.77000, loss=0.342411
2020-07-07 00:08:42,022 train INFO: validate at epoch 454, top1=93.84000, top5=99.76000, loss=0.339466
2020-07-07 00:09:21,121 train INFO: validate at epoch 455, top1=93.93000, top5=99.75000, loss=0.341604
2020-07-07 00:09:59,995 train INFO: validate at epoch 456, top1=93.96000, top5=99.75000, loss=0.340019
2020-07-07 00:10:38,458 train INFO: validate at epoch 457, top1=93.93000, top5=99.75000, loss=0.337961
2020-07-07 00:11:17,992 train INFO: validate at epoch 458, top1=93.95000, top5=99.76000, loss=0.342530
2020-07-07 00:11:54,473 train INFO: validate at epoch 459, top1=93.96000, top5=99.77000, loss=0.342851
2020-07-07 00:12:33,291 train INFO: validate at epoch 460, top1=93.91000, top5=99.76000, loss=0.341699
2020-07-07 00:13:12,189 train INFO: validate at epoch 461, top1=93.95000, top5=99.79000, loss=0.339881
2020-07-07 00:13:51,830 train INFO: validate at epoch 462, top1=93.92000, top5=99.76000, loss=0.340205
2020-07-07 00:14:30,650 train INFO: validate at epoch 463, top1=93.96000, top5=99.76000, loss=0.340678
2020-07-07 00:15:09,312 train INFO: validate at epoch 464, top1=93.92000, top5=99.77000, loss=0.340129
2020-07-07 00:15:48,737 train INFO: validate at epoch 465, top1=93.90000, top5=99.76000, loss=0.341215
2020-07-07 00:16:26,124 train INFO: validate at epoch 466, top1=93.88000, top5=99.76000, loss=0.341054
2020-07-07 00:16:56,783 train INFO: validate at epoch 467, top1=94.02000, top5=99.74000, loss=0.338008
2020-07-07 00:17:31,981 train INFO: validate at epoch 468, top1=93.92000, top5=99.75000, loss=0.343770
2020-07-07 00:18:11,246 train INFO: validate at epoch 469, top1=94.01000, top5=99.76000, loss=0.342683
2020-07-07 00:18:49,888 train INFO: validate at epoch 470, top1=93.90000, top5=99.75000, loss=0.339793
2020-07-07 00:19:29,194 train INFO: validate at epoch 471, top1=93.98000, top5=99.76000, loss=0.343511
2020-07-07 00:20:08,548 train INFO: validate at epoch 472, top1=93.96000, top5=99.75000, loss=0.345350
2020-07-07 00:20:46,214 train INFO: validate at epoch 473, top1=94.00000, top5=99.74000, loss=0.344670
2020-07-07 00:21:23,752 train INFO: validate at epoch 474, top1=93.92000, top5=99.74000, loss=0.342092
2020-07-07 00:21:56,873 train INFO: validate at epoch 475, top1=94.00000, top5=99.77000, loss=0.342705
2020-07-07 00:22:35,960 train INFO: validate at epoch 476, top1=93.92000, top5=99.75000, loss=0.339789
2020-07-07 00:23:15,198 train INFO: validate at epoch 477, top1=93.97000, top5=99.76000, loss=0.346185
2020-07-07 00:23:54,237 train INFO: validate at epoch 478, top1=93.97000, top5=99.75000, loss=0.338913
2020-07-07 00:24:32,304 train INFO: validate at epoch 479, top1=93.90000, top5=99.75000, loss=0.340938
2020-07-07 00:25:11,615 train INFO: validate at epoch 480, top1=94.05000, top5=99.75000, loss=0.340382
2020-07-07 00:25:50,218 train INFO: validate at epoch 481, top1=94.06000, top5=99.76000, loss=0.344040
2020-07-07 00:26:27,826 train INFO: validate at epoch 482, top1=94.00000, top5=99.76000, loss=0.344806
2020-07-07 00:27:05,838 train INFO: validate at epoch 483, top1=93.87000, top5=99.73000, loss=0.341880
2020-07-07 00:27:44,995 train INFO: validate at epoch 484, top1=94.02000, top5=99.77000, loss=0.342485
2020-07-07 00:28:24,483 train INFO: validate at epoch 485, top1=93.96000, top5=99.77000, loss=0.339608
2020-07-07 00:29:03,387 train INFO: validate at epoch 486, top1=93.97000, top5=99.74000, loss=0.343102
2020-07-07 00:29:42,455 train INFO: validate at epoch 487, top1=94.00000, top5=99.75000, loss=0.339408
2020-07-07 00:30:20,686 train INFO: validate at epoch 488, top1=94.02000, top5=99.76000, loss=0.342492
2020-07-07 00:30:59,647 train INFO: validate at epoch 489, top1=93.96000, top5=99.76000, loss=0.343328
2020-07-07 00:31:36,598 train INFO: validate at epoch 490, top1=93.96000, top5=99.77000, loss=0.346813
2020-07-07 00:32:15,482 train INFO: validate at epoch 491, top1=94.03000, top5=99.78000, loss=0.345640
2020-07-07 00:32:54,791 train INFO: validate at epoch 492, top1=93.96000, top5=99.75000, loss=0.345267
2020-07-07 00:33:34,044 train INFO: validate at epoch 493, top1=93.94000, top5=99.73000, loss=0.344672
2020-07-07 00:34:13,326 train INFO: validate at epoch 494, top1=93.99000, top5=99.74000, loss=0.346303
2020-07-07 00:34:52,599 train INFO: validate at epoch 495, top1=93.92000, top5=99.74000, loss=0.344240
2020-07-07 00:35:31,371 train INFO: validate at epoch 496, top1=94.00000, top5=99.73000, loss=0.343858
2020-07-07 00:36:09,574 train INFO: validate at epoch 497, top1=93.95000, top5=99.74000, loss=0.347237
2020-07-07 00:36:46,262 train INFO: validate at epoch 498, top1=93.98000, top5=99.74000, loss=0.348123
2020-07-07 00:37:25,191 train INFO: validate at epoch 499, top1=93.93000, top5=99.74000, loss=0.343327
2020-07-07 00:38:04,216 train INFO: validate at epoch 500, top1=93.94000, top5=99.73000, loss=0.346076
2020-07-07 00:38:43,362 train INFO: validate at epoch 501, top1=93.96000, top5=99.74000, loss=0.346554
2020-07-07 00:39:22,331 train INFO: validate at epoch 502, top1=93.99000, top5=99.73000, loss=0.347515
2020-07-07 00:40:00,552 train INFO: validate at epoch 503, top1=93.92000, top5=99.72000, loss=0.345395
2020-07-07 00:40:39,311 train INFO: validate at epoch 504, top1=94.05000, top5=99.73000, loss=0.344694
2020-07-07 00:41:17,532 train INFO: validate at epoch 505, top1=93.92000, top5=99.71000, loss=0.346945
2020-07-07 00:41:53,428 train INFO: validate at epoch 506, top1=93.91000, top5=99.72000, loss=0.342473
2020-07-07 00:42:32,652 train INFO: validate at epoch 507, top1=93.96000, top5=99.73000, loss=0.348046
2020-07-07 00:43:12,055 train INFO: validate at epoch 508, top1=94.02000, top5=99.75000, loss=0.347147
2020-07-07 00:43:51,131 train INFO: validate at epoch 509, top1=94.09000, top5=99.74000, loss=0.345133
2020-07-07 00:44:29,356 train INFO: validate at epoch 510, top1=94.03000, top5=99.76000, loss=0.343284
2020-07-07 00:45:08,358 train INFO: validate at epoch 511, top1=93.96000, top5=99.77000, loss=0.345734
2020-07-07 00:45:47,193 train INFO: validate at epoch 512, top1=94.07000, top5=99.74000, loss=0.347545
2020-07-07 00:46:25,462 train INFO: validate at epoch 513, top1=94.05000, top5=99.77000, loss=0.345301
2020-07-07 00:47:02,732 train INFO: validate at epoch 514, top1=93.99000, top5=99.76000, loss=0.349173
2020-07-07 00:47:41,891 train INFO: validate at epoch 515, top1=94.01000, top5=99.73000, loss=0.345514
2020-07-07 00:48:21,012 train INFO: validate at epoch 516, top1=94.05000, top5=99.74000, loss=0.347754
2020-07-07 00:48:59,908 train INFO: validate at epoch 517, top1=94.02000, top5=99.75000, loss=0.342194
2020-07-07 00:49:38,670 train INFO: validate at epoch 518, top1=93.98000, top5=99.70000, loss=0.348123
2020-07-07 00:50:17,303 train INFO: validate at epoch 519, top1=94.01000, top5=99.73000, loss=0.346920
2020-07-07 00:50:56,173 train INFO: validate at epoch 520, top1=93.94000, top5=99.77000, loss=0.353206
2020-07-07 00:51:33,086 train INFO: validate at epoch 521, top1=94.03000, top5=99.74000, loss=0.349091
2020-07-07 00:52:12,279 train INFO: validate at epoch 522, top1=94.00000, top5=99.75000, loss=0.346936
2020-07-07 00:52:51,873 train INFO: validate at epoch 523, top1=93.96000, top5=99.70000, loss=0.344318
2020-07-07 00:53:31,290 train INFO: validate at epoch 524, top1=93.94000, top5=99.73000, loss=0.344999
2020-07-07 00:54:09,514 train INFO: validate at epoch 525, top1=94.01000, top5=99.74000, loss=0.349580
2020-07-07 00:54:47,887 train INFO: validate at epoch 526, top1=94.05000, top5=99.76000, loss=0.347306
2020-07-07 00:55:26,836 train INFO: validate at epoch 527, top1=93.96000, top5=99.75000, loss=0.347229
2020-07-07 00:56:06,080 train INFO: validate at epoch 528, top1=93.91000, top5=99.74000, loss=0.347404
2020-07-07 00:56:42,410 train INFO: validate at epoch 529, top1=93.88000, top5=99.74000, loss=0.347374
2020-07-07 00:57:21,875 train INFO: validate at epoch 530, top1=93.99000, top5=99.75000, loss=0.347590
2020-07-07 00:58:01,017 train INFO: validate at epoch 531, top1=94.00000, top5=99.72000, loss=0.344218
2020-07-07 00:58:40,431 train INFO: validate at epoch 532, top1=93.98000, top5=99.74000, loss=0.345965
2020-07-07 00:59:19,742 train INFO: validate at epoch 533, top1=93.96000, top5=99.73000, loss=0.350793
2020-07-07 00:59:58,878 train INFO: validate at epoch 534, top1=93.97000, top5=99.72000, loss=0.347909
2020-07-07 01:00:37,902 train INFO: validate at epoch 535, top1=93.95000, top5=99.72000, loss=0.348393
2020-07-07 01:01:17,696 train INFO: validate at epoch 536, top1=93.92000, top5=99.75000, loss=0.346586
2020-07-07 01:01:53,728 train INFO: validate at epoch 537, top1=93.98000, top5=99.74000, loss=0.347725
2020-07-07 01:02:32,567 train INFO: validate at epoch 538, top1=93.99000, top5=99.72000, loss=0.346428
2020-07-07 01:03:11,529 train INFO: validate at epoch 539, top1=93.99000, top5=99.72000, loss=0.350038
2020-07-07 01:03:50,044 train INFO: validate at epoch 540, top1=93.97000, top5=99.74000, loss=0.347403
2020-07-07 01:04:28,738 train INFO: validate at epoch 541, top1=93.91000, top5=99.72000, loss=0.351244
2020-07-07 01:05:07,015 train INFO: validate at epoch 542, top1=93.99000, top5=99.75000, loss=0.350219
2020-07-07 01:05:45,872 train INFO: validate at epoch 543, top1=93.99000, top5=99.75000, loss=0.352056
2020-07-07 01:06:23,927 train INFO: validate at epoch 544, top1=93.99000, top5=99.72000, loss=0.347378
2020-07-07 01:07:01,267 train INFO: validate at epoch 545, top1=94.05000, top5=99.72000, loss=0.350205
2020-07-07 01:07:40,259 train INFO: validate at epoch 546, top1=93.99000, top5=99.74000, loss=0.347510
2020-07-07 01:08:19,090 train INFO: validate at epoch 547, top1=93.97000, top5=99.74000, loss=0.349828
2020-07-07 01:08:58,023 train INFO: validate at epoch 548, top1=93.93000, top5=99.72000, loss=0.348529
2020-07-07 01:09:36,746 train INFO: validate at epoch 549, top1=93.96000, top5=99.70000, loss=0.349817
2020-07-07 01:10:15,695 train INFO: validate at epoch 550, top1=93.93000, top5=99.72000, loss=0.351198
2020-07-07 01:10:55,071 train INFO: validate at epoch 551, top1=93.96000, top5=99.72000, loss=0.347680
2020-07-07 01:11:32,145 train INFO: validate at epoch 552, top1=93.99000, top5=99.70000, loss=0.348365
2020-07-07 01:12:11,052 train INFO: validate at epoch 553, top1=94.00000, top5=99.73000, loss=0.348108
2020-07-07 01:12:50,186 train INFO: validate at epoch 554, top1=93.95000, top5=99.73000, loss=0.349801
2020-07-07 01:13:29,182 train INFO: validate at epoch 555, top1=93.96000, top5=99.72000, loss=0.350004
2020-07-07 01:14:07,818 train INFO: validate at epoch 556, top1=94.01000, top5=99.73000, loss=0.350060
2020-07-07 01:14:46,815 train INFO: validate at epoch 557, top1=93.90000, top5=99.71000, loss=0.351837
2020-07-07 01:15:25,524 train INFO: validate at epoch 558, top1=93.92000, top5=99.71000, loss=0.350546
2020-07-07 01:16:04,703 train INFO: validate at epoch 559, top1=94.04000, top5=99.74000, loss=0.350311
2020-07-07 01:16:39,028 train INFO: validate at epoch 560, top1=93.97000, top5=99.70000, loss=0.348055
2020-07-07 01:17:10,085 train INFO: validate at epoch 561, top1=93.95000, top5=99.73000, loss=0.347681
2020-07-07 01:17:50,118 train INFO: validate at epoch 562, top1=93.96000, top5=99.71000, loss=0.348035
2020-07-07 01:18:29,370 train INFO: validate at epoch 563, top1=94.09000, top5=99.72000, loss=0.349979
2020-07-07 01:19:08,754 train INFO: validate at epoch 564, top1=93.99000, top5=99.73000, loss=0.347324
2020-07-07 01:19:47,592 train INFO: validate at epoch 565, top1=93.98000, top5=99.72000, loss=0.347971
2020-07-07 01:20:26,791 train INFO: validate at epoch 566, top1=93.97000, top5=99.72000, loss=0.350264
2020-07-07 01:21:05,862 train INFO: validate at epoch 567, top1=94.00000, top5=99.69000, loss=0.347945
2020-07-07 01:21:40,310 train INFO: validate at epoch 568, top1=93.96000, top5=99.71000, loss=0.352100
2020-07-07 01:22:19,062 train INFO: validate at epoch 569, top1=93.97000, top5=99.74000, loss=0.350016
2020-07-07 01:22:58,321 train INFO: validate at epoch 570, top1=94.01000, top5=99.72000, loss=0.346705
2020-07-07 01:23:37,409 train INFO: validate at epoch 571, top1=93.95000, top5=99.72000, loss=0.349372
2020-07-07 01:24:16,414 train INFO: validate at epoch 572, top1=93.92000, top5=99.71000, loss=0.352472
2020-07-07 01:24:54,785 train INFO: validate at epoch 573, top1=94.04000, top5=99.73000, loss=0.348185
2020-07-07 01:25:33,541 train INFO: validate at epoch 574, top1=94.04000, top5=99.71000, loss=0.352025
2020-07-07 01:26:12,031 train INFO: validate at epoch 575, top1=93.99000, top5=99.75000, loss=0.349485
2020-07-07 01:26:48,983 train INFO: validate at epoch 576, top1=94.01000, top5=99.70000, loss=0.348832
2020-07-07 01:27:27,718 train INFO: validate at epoch 577, top1=94.01000, top5=99.72000, loss=0.349584
2020-07-07 01:28:06,841 train INFO: validate at epoch 578, top1=93.98000, top5=99.73000, loss=0.349969
2020-07-07 01:28:45,580 train INFO: validate at epoch 579, top1=93.97000, top5=99.73000, loss=0.348893
2020-07-07 01:29:24,824 train INFO: validate at epoch 580, top1=94.04000, top5=99.74000, loss=0.351927
2020-07-07 01:30:03,200 train INFO: validate at epoch 581, top1=94.05000, top5=99.71000, loss=0.351931
2020-07-07 01:30:41,002 train INFO: validate at epoch 582, top1=93.95000, top5=99.75000, loss=0.351534
2020-07-07 01:31:19,717 train INFO: validate at epoch 583, top1=94.04000, top5=99.74000, loss=0.353141
2020-07-07 01:31:56,686 train INFO: validate at epoch 584, top1=93.99000, top5=99.72000, loss=0.350321
2020-07-07 01:32:35,741 train INFO: validate at epoch 585, top1=94.06000, top5=99.71000, loss=0.351909
2020-07-07 01:33:13,516 train INFO: validate at epoch 586, top1=94.03000, top5=99.73000, loss=0.356324
2020-07-07 01:33:52,383 train INFO: validate at epoch 587, top1=94.05000, top5=99.73000, loss=0.351939
2020-07-07 01:34:31,065 train INFO: validate at epoch 588, top1=94.04000, top5=99.73000, loss=0.352444
2020-07-07 01:35:10,134 train INFO: validate at epoch 589, top1=94.01000, top5=99.74000, loss=0.351920
2020-07-07 01:35:49,674 train INFO: validate at epoch 590, top1=93.99000, top5=99.75000, loss=0.350230
2020-07-07 01:36:27,058 train INFO: validate at epoch 591, top1=94.00000, top5=99.74000, loss=0.349069
2020-07-07 01:37:04,048 train INFO: validate at epoch 592, top1=94.01000, top5=99.75000, loss=0.351235
2020-07-07 01:37:43,662 train INFO: validate at epoch 593, top1=93.94000, top5=99.72000, loss=0.350541
2020-07-07 01:38:22,450 train INFO: validate at epoch 594, top1=94.06000, top5=99.74000, loss=0.349140
2020-07-07 01:39:01,409 train INFO: validate at epoch 595, top1=93.96000, top5=99.72000, loss=0.351905
2020-07-07 01:39:40,096 train INFO: validate at epoch 596, top1=94.10000, top5=99.71000, loss=0.350359
2020-07-07 01:40:19,087 train INFO: validate at epoch 597, top1=94.07000, top5=99.75000, loss=0.347739
2020-07-07 01:40:57,965 train INFO: validate at epoch 598, top1=94.08000, top5=99.73000, loss=0.353100
2020-07-07 01:41:34,392 train INFO: validate at epoch 599, top1=94.05000, top5=99.74000, loss=0.351075
2020-07-07 01:42:13,610 train INFO: validate at epoch 600, top1=94.08000, top5=99.74000, loss=0.346093
2020-07-07 01:42:52,396 train INFO: validate at epoch 601, top1=94.05000, top5=99.73000, loss=0.351554
2020-07-07 01:43:31,485 train INFO: validate at epoch 602, top1=93.97000, top5=99.72000, loss=0.347221
2020-07-07 01:44:10,208 train INFO: validate at epoch 603, top1=94.07000, top5=99.72000, loss=0.349917
2020-07-07 01:44:48,949 train INFO: validate at epoch 604, top1=94.03000, top5=99.73000, loss=0.351449
2020-07-07 01:45:27,954 train INFO: validate at epoch 605, top1=94.13000, top5=99.73000, loss=0.352848
2020-07-07 01:46:08,476 train INFO: validate at epoch 606, top1=94.04000, top5=99.72000, loss=0.350707
2020-07-07 01:46:43,695 train INFO: validate at epoch 607, top1=94.01000, top5=99.71000, loss=0.347429
2020-07-07 01:47:22,501 train INFO: validate at epoch 608, top1=93.98000, top5=99.74000, loss=0.350227
2020-07-07 01:48:01,419 train INFO: validate at epoch 609, top1=94.02000, top5=99.74000, loss=0.347881
2020-07-07 01:48:40,264 train INFO: validate at epoch 610, top1=93.99000, top5=99.73000, loss=0.348105
2020-07-07 01:49:18,937 train INFO: validate at epoch 611, top1=94.08000, top5=99.74000, loss=0.352112
2020-07-07 01:49:56,449 train INFO: validate at epoch 612, top1=94.01000, top5=99.74000, loss=0.346525
2020-07-07 01:50:35,476 train INFO: validate at epoch 613, top1=93.96000, top5=99.73000, loss=0.348391
2020-07-07 01:51:13,732 train INFO: validate at epoch 614, top1=94.08000, top5=99.74000, loss=0.349282
2020-07-07 01:51:49,142 train INFO: validate at epoch 615, top1=93.96000, top5=99.74000, loss=0.349332
2020-07-07 01:52:27,853 train INFO: validate at epoch 616, top1=93.91000, top5=99.72000, loss=0.345964
2020-07-07 01:53:07,384 train INFO: validate at epoch 617, top1=94.07000, top5=99.73000, loss=0.348576
2020-07-07 01:53:45,706 train INFO: validate at epoch 618, top1=94.02000, top5=99.74000, loss=0.346119
2020-07-07 01:54:24,200 train INFO: validate at epoch 619, top1=94.04000, top5=99.72000, loss=0.348870
2020-07-07 01:55:02,759 train INFO: validate at epoch 620, top1=94.01000, top5=99.71000, loss=0.349902
2020-07-07 01:55:41,894 train INFO: validate at epoch 621, top1=94.02000, top5=99.71000, loss=0.351696
2020-07-07 01:56:20,691 train INFO: validate at epoch 622, top1=94.09000, top5=99.73000, loss=0.348053
2020-07-07 01:56:56,714 train INFO: validate at epoch 623, top1=94.01000, top5=99.73000, loss=0.352070
2020-07-07 01:57:35,593 train INFO: validate at epoch 624, top1=94.07000, top5=99.75000, loss=0.350112
2020-07-07 01:58:15,230 train INFO: validate at epoch 625, top1=94.05000, top5=99.75000, loss=0.349444
2020-07-07 01:58:53,629 train INFO: validate at epoch 626, top1=94.01000, top5=99.73000, loss=0.347867
2020-07-07 01:59:32,192 train INFO: validate at epoch 627, top1=94.09000, top5=99.72000, loss=0.350672
2020-07-07 02:00:10,292 train INFO: validate at epoch 628, top1=94.06000, top5=99.74000, loss=0.350303
2020-07-07 02:00:48,275 train INFO: validate at epoch 629, top1=93.96000, top5=99.69000, loss=0.349698
2020-07-07 02:01:26,774 train INFO: validate at epoch 630, top1=94.00000, top5=99.71000, loss=0.346342
2020-07-07 02:02:03,408 train INFO: validate at epoch 631, top1=94.00000, top5=99.74000, loss=0.352085
2020-07-07 02:02:42,765 train INFO: validate at epoch 632, top1=93.98000, top5=99.73000, loss=0.350693
2020-07-07 02:03:21,843 train INFO: validate at epoch 633, top1=94.00000, top5=99.71000, loss=0.346312
2020-07-07 02:04:00,795 train INFO: validate at epoch 634, top1=93.97000, top5=99.72000, loss=0.349072
2020-07-07 02:04:40,012 train INFO: validate at epoch 635, top1=94.08000, top5=99.73000, loss=0.348169
2020-07-07 02:05:19,026 train INFO: validate at epoch 636, top1=93.99000, top5=99.74000, loss=0.353867
2020-07-07 02:05:57,598 train INFO: validate at epoch 637, top1=94.02000, top5=99.73000, loss=0.350099
2020-07-07 02:06:34,220 train INFO: validate at epoch 638, top1=94.07000, top5=99.72000, loss=0.348688
2020-07-07 02:07:13,294 train INFO: validate at epoch 639, top1=94.03000, top5=99.73000, loss=0.350525
2020-07-07 02:07:52,211 train INFO: validate at epoch 640, top1=93.97000, top5=99.72000, loss=0.346186
2020-07-07 02:08:30,009 train INFO: validate at epoch 641, top1=94.05000, top5=99.71000, loss=0.347851
2020-07-07 02:09:08,386 train INFO: validate at epoch 642, top1=94.04000, top5=99.73000, loss=0.353845
2020-07-07 02:09:47,515 train INFO: validate at epoch 643, top1=94.03000, top5=99.73000, loss=0.347824
2020-07-07 02:10:26,208 train INFO: validate at epoch 644, top1=94.00000, top5=99.70000, loss=0.348507
2020-07-07 02:11:04,461 train INFO: validate at epoch 645, top1=94.01000, top5=99.71000, loss=0.350004
2020-07-07 02:11:40,317 train INFO: validate at epoch 646, top1=94.05000, top5=99.72000, loss=0.350662
2020-07-07 02:12:19,246 train INFO: validate at epoch 647, top1=94.01000, top5=99.73000, loss=0.346956
2020-07-07 02:12:58,227 train INFO: validate at epoch 648, top1=93.96000, top5=99.71000, loss=0.348197
2020-07-07 02:13:36,880 train INFO: validate at epoch 649, top1=94.01000, top5=99.72000, loss=0.347165
2020-07-07 02:14:15,882 train INFO: validate at epoch 650, top1=94.01000, top5=99.72000, loss=0.349212
2020-07-07 02:14:54,788 train INFO: validate at epoch 651, top1=94.08000, top5=99.75000, loss=0.350868
2020-07-07 02:15:34,048 train INFO: validate at epoch 652, top1=93.96000, top5=99.74000, loss=0.349797
2020-07-07 02:16:12,729 train INFO: validate at epoch 653, top1=93.96000, top5=99.74000, loss=0.346120
2020-07-07 02:16:48,129 train INFO: validate at epoch 654, top1=94.02000, top5=99.73000, loss=0.349653
2020-07-07 02:17:27,223 train INFO: validate at epoch 655, top1=94.05000, top5=99.73000, loss=0.348541
2020-07-07 02:18:06,072 train INFO: validate at epoch 656, top1=94.05000, top5=99.73000, loss=0.351201
2020-07-07 02:18:44,787 train INFO: validate at epoch 657, top1=94.00000, top5=99.69000, loss=0.349763
2020-07-07 02:19:23,535 train INFO: validate at epoch 658, top1=94.03000, top5=99.71000, loss=0.349448
2020-07-07 02:20:02,116 train INFO: validate at epoch 659, top1=93.96000, top5=99.73000, loss=0.348890
2020-07-07 02:20:40,694 train INFO: validate at epoch 660, top1=93.96000, top5=99.74000, loss=0.349487
2020-07-07 02:21:19,649 train INFO: validate at epoch 661, top1=93.97000, top5=99.73000, loss=0.348458
2020-07-07 02:21:53,348 train INFO: validate at epoch 662, top1=94.11000, top5=99.75000, loss=0.351987
2020-07-07 02:22:32,101 train INFO: validate at epoch 663, top1=94.06000, top5=99.73000, loss=0.348611
2020-07-07 02:23:10,884 train INFO: validate at epoch 664, top1=94.08000, top5=99.74000, loss=0.346616
2020-07-07 02:23:49,441 train INFO: validate at epoch 665, top1=94.00000, top5=99.72000, loss=0.349786
2020-07-07 02:24:28,633 train INFO: validate at epoch 666, top1=94.03000, top5=99.74000, loss=0.350195
2020-07-07 02:25:08,159 train INFO: validate at epoch 667, top1=94.05000, top5=99.73000, loss=0.348412
2020-07-07 02:25:47,177 train INFO: validate at epoch 668, top1=93.96000, top5=99.72000, loss=0.347643
2020-07-07 02:26:25,914 train INFO: validate at epoch 669, top1=94.02000, top5=99.74000, loss=0.351623
2020-07-07 02:27:02,726 train INFO: validate at epoch 670, top1=94.05000, top5=99.74000, loss=0.347670
2020-07-07 02:27:42,059 train INFO: validate at epoch 671, top1=94.02000, top5=99.73000, loss=0.350157
2020-07-07 02:28:20,914 train INFO: validate at epoch 672, top1=94.11000, top5=99.75000, loss=0.348863
2020-07-07 02:29:00,414 train INFO: validate at epoch 673, top1=93.89000, top5=99.73000, loss=0.352093
2020-07-07 02:29:39,100 train INFO: validate at epoch 674, top1=93.99000, top5=99.73000, loss=0.352950
2020-07-07 02:30:17,736 train INFO: validate at epoch 675, top1=94.13000, top5=99.74000, loss=0.346997
2020-07-07 02:30:56,432 train INFO: validate at epoch 676, top1=94.06000, top5=99.74000, loss=0.349677
2020-07-07 02:31:34,204 train INFO: validate at epoch 677, top1=93.99000, top5=99.72000, loss=0.349958
2020-07-07 02:32:13,191 train INFO: validate at epoch 678, top1=93.94000, top5=99.72000, loss=0.349357
2020-07-07 02:32:52,473 train INFO: validate at epoch 679, top1=93.98000, top5=99.72000, loss=0.347818
2020-07-07 02:33:31,951 train INFO: validate at epoch 680, top1=94.04000, top5=99.71000, loss=0.347254
2020-07-07 02:34:11,167 train INFO: validate at epoch 681, top1=94.01000, top5=99.73000, loss=0.348209
2020-07-07 02:34:49,616 train INFO: validate at epoch 682, top1=94.08000, top5=99.76000, loss=0.353601
2020-07-07 02:35:28,665 train INFO: validate at epoch 683, top1=94.00000, top5=99.71000, loss=0.348523
2020-07-07 02:36:07,309 train INFO: validate at epoch 684, top1=94.06000, top5=99.74000, loss=0.351984
2020-07-07 02:36:45,071 train INFO: validate at epoch 685, top1=94.05000, top5=99.73000, loss=0.349968
2020-07-07 02:37:24,329 train INFO: validate at epoch 686, top1=94.03000, top5=99.75000, loss=0.347992
2020-07-07 02:38:03,390 train INFO: validate at epoch 687, top1=94.05000, top5=99.70000, loss=0.349707
2020-07-07 02:38:42,600 train INFO: validate at epoch 688, top1=94.07000, top5=99.71000, loss=0.349465
2020-07-07 02:39:21,201 train INFO: validate at epoch 689, top1=94.03000, top5=99.73000, loss=0.348014
2020-07-07 02:39:59,088 train INFO: validate at epoch 690, top1=93.98000, top5=99.73000, loss=0.347948
2020-07-07 02:40:37,926 train INFO: validate at epoch 691, top1=94.04000, top5=99.73000, loss=0.350634
2020-07-07 02:41:17,010 train INFO: validate at epoch 692, top1=94.04000, top5=99.72000, loss=0.345629
2020-07-07 02:41:54,721 train INFO: validate at epoch 693, top1=94.02000, top5=99.72000, loss=0.351487
2020-07-07 02:42:33,942 train INFO: validate at epoch 694, top1=94.00000, top5=99.72000, loss=0.348087
2020-07-07 02:43:13,042 train INFO: validate at epoch 695, top1=93.95000, top5=99.74000, loss=0.348503
2020-07-07 02:43:52,467 train INFO: validate at epoch 696, top1=94.02000, top5=99.75000, loss=0.347796
2020-07-07 02:44:31,858 train INFO: validate at epoch 697, top1=94.03000, top5=99.72000, loss=0.344269
2020-07-07 02:45:10,819 train INFO: validate at epoch 698, top1=94.05000, top5=99.74000, loss=0.348790
2020-07-07 02:45:50,177 train INFO: validate at epoch 699, top1=93.98000, top5=99.72000, loss=0.351418
2020-07-07 02:46:28,591 train INFO: validate at epoch 700, top1=93.99000, top5=99.74000, loss=0.350449
2020-07-07 02:47:07,751 train INFO: validate at epoch 701, top1=94.04000, top5=99.72000, loss=0.350069
2020-07-07 02:47:46,506 train INFO: validate at epoch 702, top1=94.06000, top5=99.74000, loss=0.349398
2020-07-07 02:48:25,986 train INFO: validate at epoch 703, top1=94.06000, top5=99.73000, loss=0.346386
2020-07-07 02:49:05,018 train INFO: validate at epoch 704, top1=94.03000, top5=99.72000, loss=0.345886
2020-07-07 02:49:44,626 train INFO: validate at epoch 705, top1=94.10000, top5=99.74000, loss=0.351838
2020-07-07 02:50:24,052 train INFO: validate at epoch 706, top1=94.06000, top5=99.73000, loss=0.349941
2020-07-07 02:51:02,871 train INFO: validate at epoch 707, top1=93.98000, top5=99.71000, loss=0.349743
2020-07-07 02:51:39,798 train INFO: validate at epoch 708, top1=94.01000, top5=99.73000, loss=0.348164
2020-07-07 02:52:18,754 train INFO: validate at epoch 709, top1=93.98000, top5=99.71000, loss=0.347216
2020-07-07 02:52:57,612 train INFO: validate at epoch 710, top1=94.06000, top5=99.75000, loss=0.348472
2020-07-07 02:53:37,019 train INFO: validate at epoch 711, top1=94.02000, top5=99.72000, loss=0.348134
2020-07-07 02:54:16,393 train INFO: validate at epoch 712, top1=94.06000, top5=99.71000, loss=0.350077
2020-07-07 02:54:55,877 train INFO: validate at epoch 713, top1=93.96000, top5=99.71000, loss=0.350677
2020-07-07 02:55:35,220 train INFO: validate at epoch 714, top1=93.96000, top5=99.73000, loss=0.349010
2020-07-07 02:56:14,503 train INFO: validate at epoch 715, top1=93.99000, top5=99.73000, loss=0.352113
2020-07-07 02:56:51,442 train INFO: validate at epoch 716, top1=94.14000, top5=99.75000, loss=0.350466
2020-07-07 02:57:30,430 train INFO: validate at epoch 717, top1=94.05000, top5=99.73000, loss=0.349646
2020-07-07 02:58:09,548 train INFO: validate at epoch 718, top1=93.97000, top5=99.74000, loss=0.346858
2020-07-07 02:58:48,989 train INFO: validate at epoch 719, top1=94.06000, top5=99.73000, loss=0.348317
2020-07-07 02:59:28,391 train INFO: validate at epoch 720, top1=94.05000, top5=99.71000, loss=0.347900
2020-07-07 03:00:07,478 train INFO: validate at epoch 721, top1=93.96000, top5=99.73000, loss=0.345858
2020-07-07 03:00:46,430 train INFO: validate at epoch 722, top1=93.98000, top5=99.72000, loss=0.345130
2020-07-07 03:01:25,801 train INFO: validate at epoch 723, top1=94.13000, top5=99.73000, loss=0.347770
2020-07-07 03:02:03,457 train INFO: validate at epoch 724, top1=94.04000, top5=99.75000, loss=0.348561
2020-07-07 03:02:43,051 train INFO: validate at epoch 725, top1=94.00000, top5=99.70000, loss=0.348712
2020-07-07 03:03:22,244 train INFO: validate at epoch 726, top1=94.02000, top5=99.74000, loss=0.350707
2020-07-07 03:04:01,460 train INFO: validate at epoch 727, top1=94.03000, top5=99.73000, loss=0.349832
2020-07-07 03:04:40,021 train INFO: validate at epoch 728, top1=94.05000, top5=99.72000, loss=0.349041
2020-07-07 03:05:18,134 train INFO: validate at epoch 729, top1=94.01000, top5=99.72000, loss=0.352391
2020-07-07 03:05:57,071 train INFO: validate at epoch 730, top1=94.09000, top5=99.75000, loss=0.349281
2020-07-07 03:06:34,123 train INFO: validate at epoch 731, top1=94.06000, top5=99.73000, loss=0.354202
2020-07-07 03:07:13,842 train INFO: validate at epoch 732, top1=94.00000, top5=99.71000, loss=0.348054
2020-07-07 03:07:52,992 train INFO: validate at epoch 733, top1=93.97000, top5=99.74000, loss=0.345374
2020-07-07 03:08:31,911 train INFO: validate at epoch 734, top1=94.01000, top5=99.70000, loss=0.349256
2020-07-07 03:09:11,555 train INFO: validate at epoch 735, top1=94.01000, top5=99.73000, loss=0.348620
2020-07-07 03:09:50,376 train INFO: validate at epoch 736, top1=93.97000, top5=99.72000, loss=0.347348
2020-07-07 03:10:28,631 train INFO: validate at epoch 737, top1=94.06000, top5=99.72000, loss=0.351877
2020-07-07 03:11:06,353 train INFO: validate at epoch 738, top1=93.99000, top5=99.75000, loss=0.350578
2020-07-07 03:11:43,179 train INFO: validate at epoch 739, top1=94.03000, top5=99.75000, loss=0.354149
2020-07-07 03:12:21,909 train INFO: validate at epoch 740, top1=93.93000, top5=99.74000, loss=0.347920
2020-07-07 03:13:00,561 train INFO: validate at epoch 741, top1=94.05000, top5=99.74000, loss=0.349676
2020-07-07 03:13:38,319 train INFO: validate at epoch 742, top1=94.05000, top5=99.72000, loss=0.350385
2020-07-07 03:14:16,987 train INFO: validate at epoch 743, top1=94.04000, top5=99.74000, loss=0.348082
2020-07-07 03:14:56,077 train INFO: validate at epoch 744, top1=94.06000, top5=99.73000, loss=0.348523
2020-07-07 03:15:34,967 train INFO: validate at epoch 745, top1=93.94000, top5=99.72000, loss=0.345697
2020-07-07 03:16:13,418 train INFO: validate at epoch 746, top1=94.02000, top5=99.74000, loss=0.348811
2020-07-07 03:16:49,364 train INFO: validate at epoch 747, top1=94.02000, top5=99.74000, loss=0.349940
2020-07-07 03:17:28,586 train INFO: validate at epoch 748, top1=94.06000, top5=99.74000, loss=0.346689
2020-07-07 03:18:06,965 train INFO: validate at epoch 749, top1=94.05000, top5=99.75000, loss=0.349078
2020-07-07 03:18:45,902 train INFO: validate at epoch 750, top1=94.02000, top5=99.70000, loss=0.345961
2020-07-07 03:19:24,492 train INFO: validate at epoch 751, top1=94.00000, top5=99.74000, loss=0.347981
2020-07-07 03:20:04,100 train INFO: validate at epoch 752, top1=94.07000, top5=99.74000, loss=0.348473
2020-07-07 03:20:43,332 train INFO: validate at epoch 753, top1=94.08000, top5=99.73000, loss=0.352061
2020-07-07 03:21:22,423 train INFO: validate at epoch 754, top1=94.05000, top5=99.71000, loss=0.349307
2020-07-07 03:21:58,831 train INFO: validate at epoch 755, top1=94.08000, top5=99.73000, loss=0.355893
2020-07-07 03:22:37,923 train INFO: validate at epoch 756, top1=93.98000, top5=99.73000, loss=0.349170
2020-07-07 03:23:16,798 train INFO: validate at epoch 757, top1=93.99000, top5=99.73000, loss=0.349376
2020-07-07 03:23:55,518 train INFO: validate at epoch 758, top1=94.13000, top5=99.74000, loss=0.348928
2020-07-07 03:24:33,689 train INFO: validate at epoch 759, top1=94.04000, top5=99.75000, loss=0.351674
2020-07-07 03:25:11,624 train INFO: validate at epoch 760, top1=94.02000, top5=99.75000, loss=0.351047
2020-07-07 03:25:51,152 train INFO: validate at epoch 761, top1=94.05000, top5=99.74000, loss=0.350024
2020-07-07 03:26:29,361 train INFO: validate at epoch 762, top1=94.00000, top5=99.72000, loss=0.349469
2020-07-07 03:27:07,471 train INFO: validate at epoch 763, top1=94.05000, top5=99.72000, loss=0.348275
2020-07-07 03:27:46,026 train INFO: validate at epoch 764, top1=94.13000, top5=99.73000, loss=0.349925
2020-07-07 03:28:24,597 train INFO: validate at epoch 765, top1=94.00000, top5=99.71000, loss=0.350327
2020-07-07 03:29:02,962 train INFO: validate at epoch 766, top1=94.09000, top5=99.72000, loss=0.349654
2020-07-07 03:29:41,082 train INFO: validate at epoch 767, top1=94.06000, top5=99.72000, loss=0.348233
2020-07-07 03:30:19,796 train INFO: validate at epoch 768, top1=94.13000, top5=99.75000, loss=0.348101
2020-07-07 03:30:57,991 train INFO: validate at epoch 769, top1=94.04000, top5=99.75000, loss=0.349524
2020-07-07 03:31:33,847 train INFO: validate at epoch 770, top1=94.12000, top5=99.74000, loss=0.352478
2020-07-07 03:32:10,604 train INFO: validate at epoch 771, top1=94.04000, top5=99.74000, loss=0.346958
2020-07-07 03:32:49,925 train INFO: validate at epoch 772, top1=94.01000, top5=99.72000, loss=0.351017
2020-07-07 03:33:28,947 train INFO: validate at epoch 773, top1=94.06000, top5=99.73000, loss=0.349009
2020-07-07 03:34:08,050 train INFO: validate at epoch 774, top1=93.96000, top5=99.71000, loss=0.346200
2020-07-07 03:34:46,806 train INFO: validate at epoch 775, top1=94.10000, top5=99.73000, loss=0.347586
2020-07-07 03:35:25,769 train INFO: validate at epoch 776, top1=93.99000, top5=99.70000, loss=0.346489
2020-07-07 03:36:04,923 train INFO: validate at epoch 777, top1=93.98000, top5=99.73000, loss=0.348723
2020-07-07 03:36:40,312 train INFO: validate at epoch 778, top1=94.07000, top5=99.73000, loss=0.350376
2020-07-07 03:37:19,719 train INFO: validate at epoch 779, top1=93.91000, top5=99.74000, loss=0.348500
2020-07-07 03:37:58,959 train INFO: validate at epoch 780, top1=94.02000, top5=99.74000, loss=0.349346
2020-07-07 03:38:38,165 train INFO: validate at epoch 781, top1=94.06000, top5=99.72000, loss=0.346792
2020-07-07 03:39:16,458 train INFO: validate at epoch 782, top1=94.08000, top5=99.73000, loss=0.350329
2020-07-07 03:39:55,112 train INFO: validate at epoch 783, top1=94.11000, top5=99.75000, loss=0.344982
2020-07-07 03:40:34,022 train INFO: validate at epoch 784, top1=94.04000, top5=99.72000, loss=0.351984
2020-07-07 03:41:12,144 train INFO: validate at epoch 785, top1=94.05000, top5=99.74000, loss=0.347377
2020-07-07 03:41:47,918 train INFO: validate at epoch 786, top1=94.10000, top5=99.75000, loss=0.347370
2020-07-07 03:42:26,924 train INFO: validate at epoch 787, top1=94.01000, top5=99.73000, loss=0.348598
2020-07-07 03:43:05,962 train INFO: validate at epoch 788, top1=93.99000, top5=99.73000, loss=0.348736
2020-07-07 03:43:44,878 train INFO: validate at epoch 789, top1=94.02000, top5=99.72000, loss=0.347845
2020-07-07 03:44:23,512 train INFO: validate at epoch 790, top1=94.06000, top5=99.72000, loss=0.349412
2020-07-07 03:45:02,800 train INFO: validate at epoch 791, top1=94.03000, top5=99.73000, loss=0.349860
2020-07-07 03:45:41,891 train INFO: validate at epoch 792, top1=94.09000, top5=99.75000, loss=0.351970
2020-07-07 03:46:20,601 train INFO: validate at epoch 793, top1=94.01000, top5=99.72000, loss=0.348560
2020-07-07 03:46:55,406 train INFO: validate at epoch 794, top1=93.99000, top5=99.75000, loss=0.350278
2020-07-07 03:47:34,266 train INFO: validate at epoch 795, top1=93.97000, top5=99.74000, loss=0.349187
2020-07-07 03:48:13,661 train INFO: validate at epoch 796, top1=94.06000, top5=99.73000, loss=0.346971
2020-07-07 03:48:52,354 train INFO: validate at epoch 797, top1=94.06000, top5=99.75000, loss=0.349450
2020-07-07 03:49:31,181 train INFO: validate at epoch 798, top1=93.97000, top5=99.73000, loss=0.345612
2020-07-07 03:50:09,627 train INFO: validate at epoch 799, top1=94.07000, top5=99.73000, loss=0.349404
2020-07-07 03:50:45,166 train INFO: TRAIN speed: from 31200 to 62400 iterations, batch_size=128, examples=3993472, total_net_time=1089.9159, examples/sec=3664.0184254152246
2020-07-07 03:50:45,321 train INFO: num of params in hdf5=860083

Why use this function, croplayer

Why should the picture be cut? Isn't the picture information lost

About BN layer

@ShawnDing1994 @DingXiaoH
Hi friend,

That's a pretty impressive work.

I wanna put the AC block into my own network, it's a ResNet-like architecture. I have noticed there is BN layer after each kernel (square, horizontal and vertical), and then add those three together to get the result of AC block.

So I have questions, do I still need add one more BN layer after the result? eg. When define ResNet, does it still need to add a BN layer in bottleneck?

weights translation

Hi，I want to know when you translate trained model to inference model，how to process activation function？like ReLU.
For example：
if the 3 values(of 3 branchs) fed to Relu is -1,1,0, the values after ReLU are 1,1,0, and sum output is 2.
if first sum 3 values，and then process it，it maybe ReLU(-1 + 1 + 0)=0.
This will make the two unequal.
Thanks.

ACBlock the diff deploy and train?

thanks you jobs, when i see the ACBlock and has some fusion. the train phase it has BN and no baise , while the deploy phase it has no BN and baise, when the weight fusion ,how can the parameter match ?

Hello, is there any implementation of keras version?

The problem as above.

Is there any news about AsymConvBlock?

resnet50+acb

The author, I used the Acb module in resnet50, and the training improvement was super slow. Only 10 epochs increased by 2 percentage points, and many epochs were increased. Moreover, the distance to the target was still 20 percentage points. I wonder if there was any fitting @DingXiaoH

in the function BNReLUConv2d, is there no BN layer in the ACBlock?

def BNReLUConv2d(self, in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, padding_mode='zeros', use_original_conv=False):
if use_original_conv or kernel_size == 1 or kernel_size == (1, 1):
return super(ACNetBuilder, self).BNReLUConv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size, stride=stride,
padding=padding, dilation=dilation, groups=groups, padding_mode=padding_mode, use_original_conv=True)
bn_layer = self.BatchNorm2d(num_features=in_channels)
conv_layer = ACBlock(in_channels, out_channels, kernel_size=kernel_size, stride=stride,
padding=padding, dilation=dilation, groups=groups, padding_mode=padding_mode, deploy=self.deploy)
se = self.Sequential()
se.add_module('bn', bn_layer)
se.add_module('relu', self.ReLU())
se.add_module('acb', conv_layer)
return se
Is the bn layer in the acb?

关于acb与conv不完全等价的问题

您好！
我拜读了您的论文，结构重参数的idea很不错，工程意义很大！我关注到您在将acb-bn融合成一个普通的conv时，您好像是忽略了三个卷积层的偏置项的相加，（在代码里面我也没有看到您在将三个模块融合成一个的时候有考虑三个卷积核（square_conv、vertical_conv和horizontal_conv）的偏置项相加），所以我在测试时，用acb-bn进行预测和融合后的一个conv预测，结构会有一点点的偏差，所以请问您应该是没有考虑conv-bn中的conv的偏置b吧？

about TODOS

hi,i can not find AsymConvBlock, acnet_fuse_and_load, acnet_switch_to_deploy from acnet model.could you please release a new one ?

How to use ACNet in DenseNet

Hello, I would ask you that if the file resnet.py you write in base_model of this repository has used the Asymmetric Convolution Block?
And I want to know if I want to use Asymmetric Convolution Block in DenseNet_121, which files should I download?

Plug-in version implementation

@DingXiaoH nice work!!! I implemented a plug-in implementation about ACBlock. I hope it will help you and others

realize

The complete implementation is divided into two files:

asymmetric_convolution_block.py
conv_helper.py

file asymmetric_convolution_block.py realized ACBlock

# -*- coding: utf-8 -*-

"""
@date: 2021/2/1 下午7:10
@file: asymmetric_convolution_block.py
@author: zj
@description: 
"""

import torch.nn as nn


class AsymmetricConvolutionBlock(nn.Module):
    """
    参考[ACNet/acnet/acb.py](https://github.com/DingXiaoH/ACNet/blob/master/acnet/acb.py)实现
    """

    def __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1,
                 padding_mode='zeros', use_affine=True, reduce_gamma=False, use_last_bn=False, gamma_init=None):
        super(AsymmetricConvolutionBlock, self).__init__()

        self.in_channels = in_channels
        self.out_channels = out_channels
        self.kernel_size = kernel_size
        self.stride = stride
        self.padding = padding
        self.dilation = dilation
        self.groups = groups
        self.padding_mode = padding_mode

        self.square_conv = nn.Conv2d(in_channels=in_channels, out_channels=out_channels,
                                     kernel_size=(kernel_size, kernel_size), stride=stride,
                                     padding=padding, dilation=dilation, groups=groups, bias=False,
                                     padding_mode=padding_mode)
        self.square_bn = nn.BatchNorm2d(num_features=out_channels, affine=use_affine)

        center_offset_from_origin_border = padding - kernel_size // 2
        ver_pad_or_crop = (padding, center_offset_from_origin_border)
        hor_pad_or_crop = (center_offset_from_origin_border, padding)
        if center_offset_from_origin_border >= 0:
            self.ver_conv_crop_layer = nn.Identity()
            ver_conv_padding = ver_pad_or_crop
            self.hor_conv_crop_layer = nn.Identity()
            hor_conv_padding = hor_pad_or_crop
        else:
            self.ver_conv_crop_layer = CropLayer(crop_set=ver_pad_or_crop)
            ver_conv_padding = (0, 0)
            self.hor_conv_crop_layer = CropLayer(crop_set=hor_pad_or_crop)
            hor_conv_padding = (0, 0)
        self.ver_conv = nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=(kernel_size, 1),
                                  stride=stride,
                                  padding=ver_conv_padding, dilation=dilation, groups=groups, bias=False,
                                  padding_mode=padding_mode)

        self.hor_conv = nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=(1, kernel_size),
                                  stride=stride,
                                  padding=hor_conv_padding, dilation=dilation, groups=groups, bias=False,
                                  padding_mode=padding_mode)
        self.ver_bn = nn.BatchNorm2d(num_features=out_channels, affine=use_affine)
        self.hor_bn = nn.BatchNorm2d(num_features=out_channels, affine=use_affine)

        if reduce_gamma:
            assert not use_last_bn
            self.init_gamma(1.0 / 3)

        if use_last_bn:
            assert not reduce_gamma
            self.last_bn = nn.BatchNorm2d(num_features=out_channels, affine=True)

        if gamma_init is not None:
            assert not reduce_gamma
            self.init_gamma(gamma_init)

    def init_gamma(self, gamma_value):
        nn.init.constant_(self.square_bn.weight, gamma_value)
        nn.init.constant_(self.ver_bn.weight, gamma_value)
        nn.init.constant_(self.hor_bn.weight, gamma_value)
        print('init gamma of square, ver and hor as ', gamma_value)

    def single_init(self):
        nn.init.constant_(self.square_bn.weight, 1.0)
        nn.init.constant_(self.ver_bn.weight, 0.0)
        nn.init.constant_(self.hor_bn.weight, 0.0)
        print('init gamma of square as 1, ver and hor as 0')

    def forward(self, input):
        square_outputs = self.square_conv(input)
        square_outputs = self.square_bn(square_outputs)
        vertical_outputs = self.ver_conv_crop_layer(input)
        vertical_outputs = self.ver_conv(vertical_outputs)
        vertical_outputs = self.ver_bn(vertical_outputs)
        horizontal_outputs = self.hor_conv_crop_layer(input)
        horizontal_outputs = self.hor_conv(horizontal_outputs)
        horizontal_outputs = self.hor_bn(horizontal_outputs)
        result = square_outputs + vertical_outputs + horizontal_outputs
        if hasattr(self, 'last_bn'):
            return self.last_bn(result)
        return result


class CropLayer(nn.Module):

    #   E.g., (-1, 0) means this layer should crop the first and last rows of the feature map. And (0, -1) crops the first and last columns
    def __init__(self, crop_set):
        super(CropLayer, self).__init__()
        self.rows_to_crop = - crop_set[0]
        self.cols_to_crop = - crop_set[1]
        assert self.rows_to_crop >= 0
        assert self.cols_to_crop >= 0

    def forward(self, input):
        if self.rows_to_crop == 0 and self.cols_to_crop == 0:
            return input
        elif self.rows_to_crop > 0 and self.cols_to_crop == 0:
            return input[:, :, self.rows_to_crop:-self.rows_to_crop, :]
        elif self.rows_to_crop == 0 and self.cols_to_crop > 0:
            return input[:, :, :, self.cols_to_crop:-self.cols_to_crop]
        else:
            return input[:, :, self.rows_to_crop:-self.rows_to_crop, self.cols_to_crop:-self.cols_to_crop]

conv_helper.py used to insert and fuse ACBlock

# -*- coding: utf-8 -*-

"""
@date: 2020/12/4 下午4:11
@file: act_helper.py
@author: zj
@description: 
"""

import torch
import torch.nn as nn
from .layers.asymmetric_convolution_block import AsymmetricConvolutionBlock


def get_conv(cfg):
    """
    Args:
        cfg (CfgNode): model building configs, details are in the comments of
            the config file.
    Returns:
        nn.Module: the conv layer.
    """
    conv_type = cfg.MODEL.CONV.TYPE
    if conv_type == "Conv2d":
        return nn.Conv2d
    elif conv_type == "Conv3d":
        return nn.Conv3d
    else:
        raise NotImplementedError(
            "Conv type {} is not supported".format(conv_type)
        )


def insert_acblock(model: nn.Module):
    items = list(model.named_children())
    idx = 0
    while idx < len(items):
        name, module = items[idx]
        if isinstance(module, nn.Conv2d) and module.kernel_size[0] > 1:
            # 将标准卷积替换为ACBlock
            in_channels = module.in_channels
            out_channels = module.out_channels
            kernel_size = module.kernel_size
            stride = module.stride
            padding = module.padding
            dilation = module.dilation
            groups = module.groups
            padding_mode = module.padding_mode

            acblock = AsymmetricConvolutionBlock(in_channels,
                                                 out_channels,
                                                 kernel_size[0],
                                                 stride,
                                                 padding=padding[0],
                                                 padding_mode=padding_mode,
                                                 dilation=dilation,
                                                 groups=groups)
            model.add_module(name, acblock)
            # 如果conv层之后跟随着BN层，那么删除该BN层
            # 参考[About BN layer #35](https://github.com/DingXiaoH/ACNet/issues/35)
            if (idx + 1) < len(items) and isinstance(items[idx + 1][1], nn.BatchNorm2d):
                new_layer = nn.Identity()
                model.add_module(items[idx + 1][0], new_layer)
        else:
            insert_acblock(module)
        idx += 1


def fuse_acblock(model: nn.Module, eps=1e-5):
    for name, module in model.named_children():
        if isinstance(module, AsymmetricConvolutionBlock):
            # 将ACBlock替换为标准卷积
            # 获取NxN卷积的权重以及对应BN的权重、偏置、运行时均值、运行时方差
            square_conv_weight = module.square_conv.weight
            square_bn_weight = module.square_bn.weight
            square_bn_bias = module.square_bn.bias
            square_bn_running_mean = module.square_bn.running_mean
            square_bn_running_std = torch.sqrt(module.square_bn.running_var + eps)
            # 获取Nx1卷积的权重以及对应BN的权重、偏置、运行时均值、运行时方差
            vertical_conv_weight = module.ver_conv.weight
            vertical_bn_weight = module.ver_bn.weight
            vertical_bn_bias = module.ver_bn.bias
            vertical_bn_running_mean = module.ver_bn.running_mean
            vertical_bn_running_std = torch.sqrt(module.ver_bn.running_var + eps)
            # 获取1xN卷积的权重以及对应BN的权重、偏置、运行时均值、运行时方差
            horizontal_conv_weight = module.hor_conv.weight
            horizontal_bn_weight = module.hor_bn.weight
            horizontal_bn_bias = module.hor_bn.bias
            horizontal_bn_running_mean = module.hor_bn.running_mean
            horizontal_bn_running_std = torch.sqrt(module.hor_bn.running_var + eps)
            # 计算偏差
            fused_bias = square_bn_bias + vertical_bn_bias + horizontal_bn_bias \
                         - square_bn_running_mean * square_bn_weight / square_bn_running_std \
                         - vertical_bn_running_mean * vertical_bn_weight / vertical_bn_running_std \
                         - horizontal_bn_running_mean * horizontal_bn_weight / horizontal_bn_running_std
            # 计算权重
            fused_kernel = _fuse_kernel(square_conv_weight, square_bn_weight, square_bn_running_std)
            _add_to_square_kernel(fused_kernel,
                                  _fuse_kernel(vertical_conv_weight, vertical_bn_weight, vertical_bn_running_std))
            _add_to_square_kernel(fused_kernel,
                                  _fuse_kernel(horizontal_conv_weight, horizontal_bn_weight, horizontal_bn_running_std))
            # 新建标准卷积，赋值权重和偏差后重新插入模型
            fused_conv = nn.Conv2d(module.in_channels,
                                   module.out_channels,
                                   module.kernel_size,
                                   stride=module.stride,
                                   padding=module.padding,
                                   dilation=module.dilation,
                                   groups=module.groups,
                                   padding_mode=module.padding_mode
                                   )
            fused_conv.weight = nn.Parameter(fused_kernel)
            fused_conv.bias = nn.Parameter(fused_bias)
            model.add_module(name, fused_conv)
        else:
            fuse_acblock(module, eps=eps)


def _fuse_kernel(kernel, gamma, std):
    b_gamma = torch.reshape(gamma, (kernel.shape[0], 1, 1, 1))
    b_gamma = b_gamma.repeat(1, kernel.shape[1], kernel.shape[2], kernel.shape[3])
    b_std = torch.reshape(std, (kernel.shape[0], 1, 1, 1))
    b_std = b_std.repeat(1, kernel.shape[1], kernel.shape[2], kernel.shape[3])
    return kernel * b_gamma / b_std


def _add_to_square_kernel(square_kernel, asym_kernel):
    asym_h = asym_kernel.shape[2]
    asym_w = asym_kernel.shape[3]
    square_h = square_kernel.shape[2]
    square_w = square_kernel.shape[3]
    square_kernel[:, :, square_h // 2 - asym_h // 2: square_h // 2 - asym_h // 2 + asym_h,
    square_w // 2 - asym_w // 2: square_w // 2 - asym_w // 2 + asym_w] += asym_kernel

test

# -*- coding: utf-8 -*-

"""
@date: 2021/2/1 下午8:01
@file: test_asymmetric_convolution_block.py
@author: zj
@description: 
"""

import torch
import torch.nn as nn
from torchvision.models import resnet50

from zcls.model.layers.asymmetric_convolution_block import AsymmetricConvolutionBlock
from zcls.model.conv_helper import insert_acblock, fuse_acblock


def test_asymmetric_convolution_block():
    in_channels = 32
    out_channels = 64
    dilation = 1
    groups = 1

    # inputs == outputs
    kernel_size = 3
    stride = 1
    padding = 1
    acblock = AsymmetricConvolutionBlock(in_channels,
                                         out_channels,
                                         kernel_size,
                                         stride=stride,
                                         padding=padding,
                                         dilation=dilation,
                                         groups=groups)

    data = torch.randn(1, in_channels, 56, 56)
    outputs = acblock.forward(data)

    _, _, h, w = data.shape[:4]
    _, _, h2, w2 = outputs.shape[:4]
    assert h == h2 and w == w2

    # 下采样
    kernel_size = 3
    stride = 2
    padding = 1
    acblock = AsymmetricConvolutionBlock(in_channels,
                                         out_channels,
                                         kernel_size,
                                         stride=stride,
                                         padding=padding,
                                         dilation=dilation,
                                         groups=groups)

    data = torch.randn(1, in_channels, 56, 56)
    outputs = acblock.forward(data)

    _, _, h, w = data.shape[:4]
    _, _, h2, w2 = outputs.shape[:4]
    assert h / 2 == h2 and w / 2 == w2

    # 下采样 + 分组卷积
    kernel_size = 3
    stride = 2
    padding = 1
    groups = 8
    acblock = AsymmetricConvolutionBlock(in_channels,
                                         out_channels,
                                         kernel_size,
                                         stride=stride,
                                         padding=padding,
                                         dilation=dilation,
                                         groups=groups)

    data = torch.randn(1, in_channels, 56, 56)
    outputs = acblock.forward(data)

    _, _, h, w = data.shape[:4]
    _, _, h2, w2 = outputs.shape[:4]
    assert h / 2 == h2 and w / 2 == w2


def test_acb_helper():
    in_channels = 32
    out_channels = 64
    dilation = 1

    # 下采样 + 分组卷积
    kernel_size = 3
    stride = 2
    padding = 1
    groups = 8

    model = nn.Sequential(
        nn.Sequential(
            nn.Conv2d(in_channels, out_channels, kernel_size,
                      stride=stride, padding=padding, dilation=dilation, groups=groups),
            nn.BatchNorm2d(out_channels),
            nn.ReLU(inplace=True)
        ),
        nn.BatchNorm2d(out_channels),
        nn.ReLU(inplace=True)
    )
    model.eval()
    print(model)

    data = torch.randn(1, in_channels, 56, 56)
    insert_acblock(model)
    model.eval()
    train_outputs = model(data)
    print(model)

    fuse_acblock(model, eps=1e-5)
    model.eval()
    eval_outputs = model(data)
    print(model)

    print(torch.sum((train_outputs - eval_outputs) ** 2))
    print(torch.allclose(train_outputs, eval_outputs, atol=1e-6))
    assert torch.allclose(train_outputs, eval_outputs, atol=1e-6)


def test_resnet50_acb():
    model = resnet50()
    model.eval()
    # print(model)

    data = torch.randn(1, 3, 224, 224)
    insert_acblock(model)
    model.eval()
    train_outputs = model(data)
    # print(model)

    fuse_acblock(model, eps=1e-5)
    model.eval()
    eval_outputs = model(data)
    # print(model)

    print(torch.sum((train_outputs - eval_outputs) ** 2))
    print(torch.allclose(train_outputs, eval_outputs, atol=1e-5))
    assert torch.allclose(train_outputs, eval_outputs, atol=1e-5)


if __name__ == '__main__':
    print('*' * 10)
    test_asymmetric_convolution_block()
    print('*' * 10)
    test_acb_helper()
    print('*' * 10)
    test_resnet50_acb()
############################### output
**********
**********
Sequential(
  (0): Sequential(
    (0): Conv2d(32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=8)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
  )
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
)
Sequential(
  (0): Sequential(
    (0): AsymmetricConvolutionBlock(
      (square_conv): Conv2d(32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=8, bias=False)
      (square_bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (ver_conv_crop_layer): Identity()
      (hor_conv_crop_layer): Identity()
      (ver_conv): Conv2d(32, 64, kernel_size=(3, 1), stride=(2, 2), padding=(1, 0), groups=8, bias=False)
      (hor_conv): Conv2d(32, 64, kernel_size=(1, 3), stride=(2, 2), padding=(0, 1), groups=8, bias=False)
      (ver_bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (hor_bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (1): Identity()
    (2): ReLU(inplace=True)
  )
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
)
Sequential(
  (0): Sequential(
    (0): Conv2d(32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=8)
    (1): Identity()
    (2): ReLU(inplace=True)
  )
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
)
tensor(5.4800e-10, grad_fn=<SumBackward0>)
True
**********
tensor(1.0568e-08, grad_fn=<SumBackward0>)
True

Process finished with exit code 0

how to use

You can use other models as usual. if you want to add ACBlock, do it like this:

from zcls.model.conv_helper import insert_acblock
。。。
。。。
if cfg.MODEL.CONV.ACBLOCK is True:
    insert_acblock(model)                             # -----------------> HERE

model = model.to(device=device)
if du.get_world_size() > 1:
    model = DDP(model, device_ids=[device], output_device=device, find_unused_parameters=True)

return model

Then normal training and model parameter preservation are carried out, if you want to fuse ACBlock, you can use func fuse_acblock

The complete implementation can be referred to ZJCV/ZCls

ACNet applies to 3D convolution

hello,How to apply ACNet to 3D convolution? and does it works?

about the parameter innitialization of BatchNorm

I am a new learner of Deeplearning.I Confused by some code.

in your acnet.abc.py. what the meaning of reduce_gama and gamma_int. they seem to be two defferent innitialization method. when is the proper time or occation to do innitialization for each
innitialization method? and why the choice of the use_last_bn must be different from the choice of reduce_gama, what the accosiation between two viarables.
I am trying to understand the code and applying in my own code. Thank your for your sharing and attention

How to use for LRN?

If I replace the bn layer with lrn, could it be effective?

关于ACblock ，border的卷积核构建

大神您好，这里想问下关于ACNet论文中图6.(c)中的边缘卷积核骨架有开源的构建方式么？自己没有找到

About pretrained model

Hello, the author!
I have studied your paper and found that the model you proposed is of great significance in the classification. I intend to apply the improved-ResNet50 to my model. However, due to the limitation of computational force, I could not obtain the pre-trained weight of the improved ResNet50 (i.e. ResNet50 with asymmetric convolution block) on ImageNet. Could you share it with us? Thank you very much!

关于ACB的用法

请问，您是把ACB 嵌入到网络的neck部分吗？
谢谢

请问如何应用到resnet里？

@DingXiaoH @ShawnDing1994 作者你好，请问怎么将ACNet应用到ResNet里，需要用到哪些files？问题有些基础，先谢谢回复了！

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.