1chimarugin / lightweight-neural-architecture-search Goto Github PK

• ☀️ Hiring research interns for Neural Architecture Search, Tiny Machine Learning, Computer Vision tasks: [email protected]
• 💥 2022.09: Mixed-Precision Quantization is now supported! The QE-Score paper is accepted by NeurIPS'22.
• 💥 2022.06: Code for MAE-DET is now released.
• 💥 2022.05: MAE-DET is accepted by ICML'22.
• 💥 2021.09: Code for Zen-NAS is now released.
• 💥 2021.07: The inspiring training-free paper Zen-NAS has been accepted by ICCV'21.

English | 简体中文

Lightweight Neural Architecture Search (Light-NAS) is an open source zero-short NAS toolbox for backbone search based on PyTorch. The master branch works with PyTorch 1.4+ and OpenMPI 4.0+.

This project is developed by Alibaba and licensed under the Apache 2.0 license.

This product contains third-party components under other open source licenses.

See the NOTICE file for more information.

1.0.2 was released in 2022/09/19:

• Support madnas score for training-free search.
• Support mixed-precision Quantization tasks.
• Update the searched mixed-precision models.

Please refer to changelog.md for details and release history.

• Linux
• GCC 7+
• OpenMPI 4.0+
• Python 3.6+
• PyTorch 1.4+
• CUDA 10.0+

1. Compile the OpenMPI 4.0+ Downloads.

   cd path
   tar -xzvf openmpi-4.0.1.tar.gz
   cd openmpi-4.0.1
   ./configure --prefix=/your_path/openmpi
   make && make install

   add the commands into your ~/.bashrc

   export PATH=/your_path/openmpi/bin:$PATH
   export LD_LIBRARYPATH=/your_path/openmpi/lib:$LD_LIBRARY_PATH

2. Create a conda virtual environment and activate it.

   conda create -n light-nas python=3.6 -y
   conda activate light-nas

3. Install torch and torchvision with the following command or offcial instruction.

   pip install torch==1.4.0+cu100 torchvision==0.5.0+cu100 -f https://download.pytorch.org/whl/torch_stable.html

   if meet "Not be found for jpeg", please install the libjpeg for the system.

   sudo yum install libjpeg # for centos
   sudo apt install libjpeg-dev # for ubuntu

4. Install other packages with the following command.

   pip install -r requirements.txt

• Search with examples

  cd scripts/classification
  sh example_xxxx.sh

Note: If you find this useful, please support us by citing it.

@inproceedings{zennas,
	title     = {Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition},
	author    = {Ming Lin and Pichao Wang and Zhenhong Sun and Hesen Chen and Xiuyu Sun and Qi Qian and Hao Li and Rong Jin},
	booktitle = {2021 IEEE/CVF International Conference on Computer Vision},  
	year      = {2021},
}

The official code for Zen-NAS was originally released at https://github.com/idstcv/ZenNAS.

Note: If you find this useful, please support us by citing it.

@inproceedings{maedet,
  title     = {MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection},
  author    = {Zhenhong Sun and Ming Lin and Xiuyu Sun and Zhiyu Tan and Hao Li and Rong Jin},
  booktitle = {International Conference on Machine Learning},
  year      = {2022},
}

The ImageNet training pipeline can be found at https://github.com/tinyvision/imagenet-training-pipeline

Note: If you find this useful, please support us by citing it.

@inproceedings{qescore,
  title     = {Entropy-Driven Mixed-Precision Quantization for Deep Network Design on IoT Devices},
  author    = {Zhenhong Sun and Ce Ge and Junyan Wang and Ming Lin and Hesen Chen and Hao Li and Xiuyu Sun},
  journal   = {Advances in Neural Information Processing Systems},
  year      = {2022},
}

Zhenhong Sun, Ming Lin, Xiuyu Sun, Hesen Chen, Ce Ge.