Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
Code repository for Quantized NAS (Chapter 3): https://arxiv.org/abs/1904.09872
Installation
We recommend using virtual environment. Installing instructions can be found in the following link: https://www.tensorflow.org/install/pip
After the virtual environment activation, we have to install the required packages:
pip install -r requirements.txt
Make sure the current directory is the repository main directory.
Datasets
We worked with CIFAR-10 and CIFAR-100.
Both can be automatically downloaded by torchvision.
Usage
Search
To carry out quantized search, use the following command:
PYTHONPATH=../ CUDA_VISIBLE_DEVICES=0 python3 ./train_search.py --data ../data/ --dataset cifar10 --batch_size 250 --arch_learning_rate 0.1 --learning_rate 0.01 --lmbda 1 --bitwidth 2#2,4#3#8 --baselineBits 3 --epochs 1 --model thin_resnet --nCopies 1 --grad_estimator layer_same_path --alphas_regime alphas_weights_loop --nSamples 3 --workers 2 --train_portion 0.5 --gpu 0 --alphas_data_parts 4 --pre_trained "../pre_trained/cifar10/train_portion_1.0/[(32, 32)],[thin_resnet]/model.updated_stats.pth.tar"
Make sure the current directory is the cnn directory.
Checkpoint evaluation
During the search, we sample configurations from the current distribution. Use the following command in order to train the sampled configurations and evaluate their quality.
PYTHONPATH=../ CUDA_VISIBLE_DEVICES=0 python3 ./train_opt2.py --data ../data/ --json results/checkpoints/20190501-121257-1-4.json
Make sure the current directory is the cnn directory.
The argument --json holds the path to the checkpoint we would like to train.
Acknowledgments
The research was funded by ERC StG RAPID.
Citation
If our work helped you in your research, please consider cite us.
@ARTICLE{2019arXiv190409872Z,
author = {{Zur}, Yochai and {Baskin}, Chaim and {Zheltonozhskii}, Evgenii and
{Chmiel}, Brian and {Evron}, Itay and {Bronstein}, Alex M. and
{Mendelson}, Avi},
title = "{Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks}",
journal = {arXiv e-prints},
keywords = {Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Neural and Evolutionary Computing},
year = "2019",
month = "Apr",
eid = {arXiv:1904.09872},
pages = {arXiv:1904.09872},
archivePrefix = {arXiv},
eprint = {1904.09872},
primaryClass = {cs.CV},
adsurl = {https://ui.adsabs.harvard.edu/abs/2019arXiv190409872Z},
adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}
This work is licensed under the Creative Commons Attribution-NonCommercial
4.0 International License. To view a copy of this license, visit
http://creativecommons.org/licenses/by-nc/4.0/ or send a letter to
Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent