Reproduce the accuracy in paper

Hello, thanks for sharing your work. We have tride the code on AS-MLP-T, and only get 81.13/81.06 in two seperate experiments. Are there any settings needed to be modified?

数据集问题

你好请问数据集存放的路径应该是怎么样的我使用/data/imagenet好像不行

论文内容问题

作者您好:
您论文提及的轴转换对应到后面采样点，我想问一下本文中这个采样点和轴转换对应是什么，换句话说本文采样点这样分布怎么理解，谢谢

Opinion for Fast PatchMerging Layer

AS-MLP/models/as_mlp.py

Lines 187 to 222 in 946848d

    
           class PatchMerging(nn.Module): 
        
               r""" Patch Merging Layer. 
        
               Args: 
        
                   input_resolution (tuple[int]): Resolution of input feature. 
        
                   dim (int): Number of input channels. 
        
                   norm_layer (nn.Module, optional): Normalization layer.  Default: nn.LayerNorm 
        
               """ 
        
               def __init__(self, input_resolution, dim, norm_layer=nn.LayerNorm): 
        
                   super().__init__() 
        
                   self.input_resolution = input_resolution 
        
                   self.dim = dim 
        
                   self.reduction = nn.Conv2d(4 * dim, 2 * dim, 1, 1, bias=False) 
        
                   self.norm = norm_layer(4 * dim) 
        
               def forward(self, x): 
        
                   """ 
        
                   x: B, H*W, C 
        
                   """ 
        
                   B, C, H, W = x.shape 
        
                   #assert L == H * W, "input feature has wrong size" 
        
                   assert H % 2 == 0 and W % 2 == 0, f"x size ({H}*{W}) are not even." 
        
                   x = x.view(B, C, H, W) 
        
                   x0 = x[:, :, 0::2, 0::2]  # B C H/2 W/2  
        
                   x1 = x[:, :, 1::2, 0::2]  # B C H/2 W/2  
        
                   x2 = x[:, :, 0::2, 1::2]  # B C H/2 W/2  
        
                   x3 = x[:, :, 1::2, 1::2]  # B C H/2 W/2  
        
                   x = torch.cat([x0, x1, x2, x3], 1)  # B 4*C H/2 W/2  
        
                   x = self.norm(x) 
        
                   x = self.reduction(x) 
        
                   return x

Thanks for sharing your work!

Why did you design PathMerging Layer normalizing features first?

I think that if we normalize features after reduce layer, we can reduce the size of model and flops.

Did you do ablation study for this layer?

For example,

 class PatchMerging(nn.Module): 
     r""" Patch Merging Layer. 
  
     Args: 
         input_resolution (tuple[int]): Resolution of input feature. 
         dim (int): Number of input channels. 
         norm_layer (nn.Module, optional): Normalization layer.  Default: nn.LayerNorm 
     """ 
  
     def __init__(self, input_resolution, dim, norm_layer=nn.LayerNorm): 
         super().__init__() 
         self.input_resolution = input_resolution 
         self.dim = dim 
         self.reduction = nn.Conv2d(4 * dim, 2 * dim, 1, 1, bias=False) 
         self.norm = norm_layer(2 * dim) 
  
     def forward(self, x): 
         """ 
         x: B, H*W, C 
         """ 
         B, C, H, W = x.shape 
         #assert L == H * W, "input feature has wrong size" 
         assert H % 2 == 0 and W % 2 == 0, f"x size ({H}*{W}) are not even." 
  
         x = x.view(B, C, H, W) 
  
         x0 = x[:, :, 0::2, 0::2]  # B C H/2 W/2  
         x1 = x[:, :, 1::2, 0::2]  # B C H/2 W/2  
         x2 = x[:, :, 0::2, 1::2]  # B C H/2 W/2  
         x3 = x[:, :, 1::2, 1::2]  # B C H/2 W/2  
         x = torch.cat([x0, x1, x2, x3], 1)  # B 4*C H/2 W/2  

         x = self.reduction(x) 
         x = self.norm(x) 
         return x

Thank you for your work!
However, I did not find your code for semantic segmentation. I am currently working on pure-MLP segmentation and it would be a huge help if you could provide the ASMLP code for semantic segmentation.

Training strategy on Imagenet

Hello! Thank you for your wonderful job!
it is said that as-mlp uses the same training strategy in Swin Transformer, I want to know to what extend the work use the same strategy ? For example, does as-mlp use Mixup、Cutmix?
I saw there exists codes for mixup in function build_loader which requires config.AUG.MIXUP, but I didn't find config.AUG in the config file for models.
I want to know the detailed training setting for your models, may you do the favor for me?

代码问题

您好，在您的论文中，AS-MLP的第一个Block是不会减少图像的分辨率的，但是在你的代码中却只是最后一个block不使用downsample，我想问一下是不是理解错了？，谢谢！

Question on the Shift CUDA implementation.

Thanks for sharing such a simple yet effective backbone work~

When I run the code, I find no noticeable runtime difference between using the shift_cuda.py and using Algorithm 1 in the paper with Pytorch itself.

Do I miss something to activate the CUDA acceleration?

In fact, I copy part of the core codes in this repo and integrate the codes to the Swin Transformer, instead of running this repo directly.

Could you please help me with this problem? Thanks~

Error when using --cache-mode part

It seems that the code cannot generate "./train_pkl/samples_bytes_0.pkl" successfully.

The first running, generate pkl, but return error:

./train_pkl/samples_bytes_0.pkl
global_rank 0 cached 0/1281167 takes 0.00s per block
global_rank 0 cached 128116/1281167 takes 21.24s per block
global_rank 0 cached 256232/1281167 takes 19.01s per block
global_rank 0 cached 384348/1281167 takes 18.36s per block
global_rank 0 cached 512464/1281167 takes 29.66s per block
global_rank 0 cached 640580/1281167 takes 35.94s per block
global_rank 0 cached 768696/1281167 takes 36.32s per block
global_rank 0 cached 896812/1281167 takes 35.54s per block
global_rank 0 cached 1024928/1281167 takes 37.19s per block
global_rank 0 cached 1153044/1281167 takes 46.55s per block
global_rank 0 cached 1281160/1281167 takes 50.39s per block
Traceback (most recent call last):
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/torch/distributed/launch.py", line 260, in
main()
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/torch/distributed/launch.py", line 256, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/ma-user/anaconda3/envs/Pytorch-1.4.0/bin/python', '-u', 'main.py', '--local_rank=0', '--cfg', 'configs/as_base_patch4_shift5_224.yaml', '--data-path', '/cache/imagenet/imagenet/', '--eval', '--resume', '/cache/model/asmlp_base_patch4_shift5_224.pth', '--moxfile', '0']' died with <Signals.SIGKILL: 9>.

The second running, pkl contain nothing, so the error occurs:

./train_pkl/samples_bytes_0.pkl
Traceback (most recent call last):
File "main.py", line 349, in
main(config)
File "main.py", line 78, in main
dataset_train, dataset_val, data_loader_train, data_loader_val, mixup_fn = build_loader(config)
File "/home/ma-user/work/AS-MLP-main/data/build.py", line 17, in build_loader
dataset_train, config.MODEL.NUM_CLASSES = build_dataset(is_train=True, config=config)
File "/home/ma-user/work/AS-MLP-main/data/build.py", line 80, in build_dataset
cache_mode=config.DATA.CACHE_MODE if is_train else 'part')
File "/home/ma-user/work/AS-MLP-main/data/cached_image_folder.py", line 250, in init
cache_mode=cache_mode)
File "/home/ma-user/work/AS-MLP-main/data/cached_image_folder.py", line 122, in init
self.init_cache()
File "/home/ma-user/work/AS-MLP-main/data/cached_image_folder.py", line 137, in init_cache
self.samples = pickle.load(handle)
EOFError: Ran out of input
Traceback (most recent call last):
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/torch/distributed/launch.py", line 260, in
main()
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/torch/distributed/launch.py", line 256, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/ma-user/anaconda3/envs/Pytorch-1.4.0/bin/python', '-u', 'main.py', '--local_rank=0', '--cfg', 'configs/as_base_patch4_shift5_224.yaml', '--data-path', '/cache/imagenet/imagenet/', '--eval', '--resume', '/cache/model/asmlp_base_patch4_shift5_224.pth', '--moxfile', '0']' returned non-zero exit status 1.

Can you provide .so compiling setting?

Like many projects, they would provide torch cuda extensions and a setup.py to compile and generate .so file so that they can be loaded and used. Could you provide such method to use as_shift op?

运行问题

运行test.sh显示Error while finding module specification for 'torch.distributed.launch'但是我terminal中是有torch的包的这个分布式训练是否还需要在config文件中做修改，cuda11.0版本

Please add paper link as well.

I arrived this repo via followings' likes. For those who like me, it would be good if paper link is included in README.
Anyway, thanks for your great effort and reports.

Sincerely,
Sungguk Cha

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/amFNsyUBvm or add me on WeChat (van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

	OpenMMLab 1.0 branch	OpenMMLab 2.0 branch
MMEngine		0.x
MMCV	1.x	2.x
MMDetection	0.x 、1.x、2.x	3.x
MMAction2	0.x	1.x
MMClassification	0.x	1.x
MMSegmentation	0.x	1.x
MMDetection3D	0.x	1.x
MMEditing	0.x	1.x
MMPose	0.x	1.x
MMDeploy	0.x	1.x
MMTracking	0.x	1.x
MMOCR	0.x	1.x
MMRazor	0.x	1.x
MMSelfSup	0.x	1.x
MMRotate	1.x	1.x
MMYOLO		0.x

Attention: please create a new virtual environment for OpenMMLab 2.0.

环境问题

cuda11 apex换成另外一个版本的可以跑吗

how can I use this code do image segmentation

Errror when using train_asmlp_b.sh

When using train_asmlp_b.sh, the above error occurs:

File "/home/ma-user/work/AS-MLP-main/models/as_mlp.py", line 165, in forward
x = shortcut + self.drop_path(x)
File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/apex/amp/wrap.py", line 58, in wrapper
return orig_fn(*new_args, **kwargs)
RuntimeError: The size of tensor a (56) must match the size of tensor b (60) at non-singleton dimension 3

Some questions about the code

Hi, I have a few questions about the dimensions in yourt code.

Is your input dimension format like B C H W, eg, 1, 3, 224, 224?
If so, I suppose the dim in layernorm you initialized is C. when I run this, I got an error like
Given normalized_shape=[2304], expected input with shape [*, 2304], but got input of size[8, 2304, 8, 8]
And also, how do you get the shape of nW*B, window_size*window_size, C after axis shift?

Thank you

Will the throughput change a lot whether using shift_cuda or just torch functions?

Do you have data about this?

Credit to involution

The CUDA kernel of basic op seems to be borrowed from https://github.com/d-li14/involution, especially implied by https://github.com/svip-lab/AS-MLP/blob/main/models/shift_cuda.py#L169.

Please consider properly giving credit to the involution repo and citing us. Thanks.

	class PatchMerging(nn.Module):
	r""" Patch Merging Layer.

	Args:
	input_resolution (tuple[int]): Resolution of input feature.
	dim (int): Number of input channels.
	norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm
	"""

	def __init__(self, input_resolution, dim, norm_layer=nn.LayerNorm):
	super().__init__()
	self.input_resolution = input_resolution
	self.dim = dim
	self.reduction = nn.Conv2d(4 * dim, 2 * dim, 1, 1, bias=False)
	self.norm = norm_layer(4 * dim)

	def forward(self, x):
	"""
	x: B, H*W, C
	"""
	B, C, H, W = x.shape
	#assert L == H * W, "input feature has wrong size"
	assert H % 2 == 0 and W % 2 == 0, f"x size ({H}*{W}) are not even."

	x = x.view(B, C, H, W)

	x0 = x[:, :, 0::2, 0::2] # B C H/2 W/2
	x1 = x[:, :, 1::2, 0::2] # B C H/2 W/2
	x2 = x[:, :, 0::2, 1::2] # B C H/2 W/2
	x3 = x[:, :, 1::2, 1::2] # B C H/2 W/2
	x = torch.cat([x0, x1, x2, x3], 1) # B 4*C H/2 W/2

	x = self.norm(x)
	x = self.reduction(x)

	return x

svip-lab / as-mlp Goto Github PK

as-mlp's People

Contributors

Stargazers

Watchers

Forkers

as-mlp's Issues

Welcome update to OpenMMLab 2.0

Recommend Projects

Recommend Topics

Recommend Org