A review of convolutional neural network architectures and their optimizations

The research advances concerning the typical architectures of convolutional neural networks (CNNs) as well as their optimizations are analyzed and elaborated in detail in this paper. This paper proposes a typical approach to classifying CNNs architecture based on modules in order to accommodate more new network architectures with multiple characteristics that make them difficult to rely on the original classification method. Through the pros and cons analysis of diverse network architectures and their performance comparisons, six types of typical CNNs architectures are analyzed and explained in detail. The CNNs architectures intrinsic characteristics is also explored. Moreover, this paper provides a comprehensive classification of network compression and accelerated network architecture optimization algorithms based on the mathematical principle of various optimization algorithms. Finally, this paper analyses the strategy of NAS algorithms, discusses the applications of CNNs, and sheds light on the challenges and prospects of the current CNNs architecture and its optimizations. The explanation of the advantages brought by optimizing different network architecture types, the basis for constructively choosing appropriate CNNs in specific designs and applications are provided. This paper will help the readers to choose constructively appropriate CNNs in specific designs and applications.