Extract Generalization Ability from Convolutional Neural Networks

With the development of deep learning, the trend of mobile artificial intelligent is emerging at a rapid pace. Over time, a series of mobile devices such as smart speakers and smartphones gradually have become the first choice of artificial intelligent landing. However, the hardware resources of mobile devices are very limited. In terms of time, memory, or energy consumption, large neural networks cannot be deployed on mobile devices. Therefore, it is necessary to explore a new compression method. Studies have shown that much redundancy exists in convolutional neural networks. This means that the neural network structure can be trimmed without affecting the accuracy. Inspired by the autoencoder, this paper presents a new compression method. The method can remove redundant neurons and convolution kernels. It extracts the generalization ability of convolutional neural networks onto smaller models. Experimental results show that the pruning rate is approximately 4x to 21x, and the speedup is approximately 2x to 5x.