Speeding-up and compression convolutional neural networks by low-rank decomposition without fine-tuning

With the rapid development of convolutional neural network (CNN), the accuracy of CNN has been significantly improved, which also brings great challenges to the deployment of mobile terminals or embedded devices with limited resources. Recently, significant achievements have been made in compressing CNN through low-rank decomposition. Unlike existing methods that use the same decomposition form and decomposition strategy with fine-tuning based on singular value decomposition (SVD), our method uses different decomposition forms for different layers, and proposes decomposition strategies without fine-tuning. We present a simple and effective scheme to compress the entire CNN, which is called cosine similarity SVD without fine-tuning. For the AlexNet, our cosine similarity algorithm of rank selection takes 84% of the time to find the rank compared with the bayesian optimization (BayesOpt) algorithm. After we tested various CNNs (AlexNet, VGG-16, VGG-19, and ResNet-50) on different data sets, experimental results show that the weight parameter drop can exceed 50% when the accuracy loss is less than 1% without fine-tuning. The floating point operations (FLOPs) drop is about 20%, and the accuracy loss is less than 1% without fine-tuning.