cas-clab / dgnet Goto Github PK

View Code? Open in Web Editor NEW

29.0 3.0 5.0 671 KB

Dynamic Dual Gating Neural Networks, ICCV 2021

License: MIT License

Python 98.62% Shell 1.38%

dgnet's Introduction

Dynamic Dual Gating Neural Networks

This repository contains the PyTorch implementation for

Dynamic Dual Gating Neural Networks
Fanrong Li, Gang Li, Xiangyu He, Jian Cheng
ICCV 2021 Oral

Getting Started

Requirements

The main requirements of this work are:

Python 3.7
PyTorch == 1.5.0
Torchvision == 0.6.0
CUDA 10.2

We recommand using conda env to setup the experimental environments.

# Create environment
conda create -n DGNet python=3.7
conda activate DGNet

# Install PyTorch & Torchvision
pip install torch==1.5.0 torchvision==0.6.0

# Clone repo
git clone https://github.com/anonymous-9800/DGNet.git ./DGNet
cd ./DGNet

# Install other requirements
pip install -r requirements.txt

Trained models

Our trained models can be found here: Google Drive. And the pretrained cifar10 models can be found here: Google Drive. Unzip and place them into the DGNet folder.

Evaluate a trained DGNet

# CIFAR-10
sh ./scripts/cifar_e.sh [ARCH] [PATH-TO-DATASET] [GPU-IDs] [PATH-TO-SAVE] [PATH-TO-TRAINED-MODEL]

# ResNet on ImageNet
sh ./scripts/imagenet_e.sh [ARCH] [PATH-TO-DATASET] [GPU-IDs] [PATH-TO-SAVE] [PATH-TO-TRAINED-MODEL]

# Example
sh ./scripts/imagenet_e.sh resdg34 [PATH-TO-DATASET] 0 imagenet/resdg34-04-e ./trained_models_cls/imagenet_results/resdg34/sparse06/resdg34_04.pth.tar

Train a DGNet

# CIFAR-10
sh ./scripts/cifar_t.sh [ARCH] [PATH-TO-DATASET] [TARGET-DENSITY] [GPU-IDs] [PATH-TO-SAVE] [PATH-TO-PRETRAINED-MODEL]

# ResNet on ImageNet
sh ./scripts/imagenet_t.sh [ARCH] [PATH-TO-DATASET] [TARGET-DENSITY] [GPU-IDs] [PATH-TO-SAVE]

# Example
sh ./scripts/imagenet_t.sh resdg34 [PATH-TO-DATASET] 0.4 0,1 imagent/resdg34-04

Main results

Model	Method	Top-1 (%)	Top-5 (%)	FLOPs	Google Drive
ResNet-18	DGNet (50%)	70.12	89.22	9.54E8	Link
ResNet-18	DGNet (60%)	69.38	88.94	7.88E8	Link
ResNet-34	DGNet (60%)	73.01	90.99	1.50E9	Link
ResNet-34	DGNet (70%)	71.95	90.46	1.21E9	Link
ResNet-50	DGNet (60%)	76.41	93.05	1.65E9	Link
ResNet-50	DGNet (70%)	75.12	92.34	1.31E9	Link
MobileNet-V2	DGNet (50%)	71.62	90.05	1.60E8	Link

Citation

If you find this project useful for your research, please use the following BibTeX entry.

@inproceedings{dgnet,
  title={Dynamic Dual Gating Neural Networks},
  author={Li, Fanrong and Li, Gang and He, Xiangyu and Cheng, Jian},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
  year={2021}
}

Contact

For any questions, feel free to contact: [email protected]

dgnet's People

Contributors

Stargazers

Watchers

Forkers

cheungbh krishnatarun smn2010 ikarosy shenghsin

dgnet's Issues

Sparsity Loss

Hi,

Will it be possible for you to share plots for sparsity loss (and bound loss)? If not plots can you share some insight on behaviors of these losses (i.e do they saturate early or oscillate too much etc.)

Resnet110's experimental results on cifar

Hi, thank you for your excellent work. Using the resnet110 network, the density is 0.4, and the result of the code running is about 93.5, which is quite different from the 94.33 in the paper. Compared with other networks, is there any difference in the settings?

Regarding FPGA implementation

Hey,

Thanks for the awesome work. Can you share your FPGA implementation methodology? Which tools did you use?

Thanks!
Shivam Aggarwal

About object detection

Hello. Thanks for your work. I wonder when will you release the code about object detection?

A question about speed measurement

Hello, I'm sorry to bother you,I'd like to ask you some questions,your paper says that it can actually accelerate CPU, could you tell me how to accelerate it in detail? The speed measured by me is nearly twice full, thank you very much for your answer.

Speedup in Table 3

Thank you for your excellent work. I sincerely want to ask you:

What is the unit of Speedup in Table 3, milliseconds?
Is there a specific baseline and new network running time?
I hope to get your answers. I would appreciate it

target budget in bound loss

Dear authors, thank you for sharing this excellent work!

I wonder why the target budget is set as sqrt(T_d) rather than T_d. In https://arxiv.org/abs/1912.03203, which you also refer to in your paper, the target budget was set as T_d.

Could you please explain a bit more about this design choice? Thanks

Applying Only Channel Gating

Hi,

Thanks for the code of your work.

Could you please suggest how can I perform the ablation study of applying only channel gating. I commented out all the lines here but I am getting constant ratio of real flops and ori flops despite setting the target density between 0.3 to 0.7. I am unable to understand if I am making any errors regarding this. Please help.

Thanks

About cifar10 baseline

Hello. Thanks for your great work.
I noticed that in your paper you mentioned you obtain a good performance in cifar10 baseline. I wonder could we access the baseline checkpoints? They are not available in your folder downloaded from google.

Inferenece for different target densities

Hello,

My question is do you train a separate model for each target density(50%, 70%)? If yes, how is it dynamic during inference?

Thank You.