Lightweight Convolutional Neural Networks (Lwt-CNN) generally refer to CNN with fewer parameters. They have raised researchers' much attention and developed rapidly over the last decade due to the advantages such as feasible embeded deployment, real-time applications and less communication overhead.

The first lightweight CNN dates back to SqueezeNet which was proposed in ICLR 2017.

The picture below shows the development of lightweight CNN over the years.

The following table summarizes the structure of the core modules in the lightweight neural networks. The usage of batch normalization (BN) and activation function (act) in these modules are particularly indicated.