LPSD: Low-Rank Plus Sparse Decomposition for Highly Compressed CNN Models

Low-rank decomposition that explores and eliminates the linear dependency within a tensor is often used as a structured model pruning method for deep convolutional neural networks. However, the model accuracy declines rapidly as the compression ratio increases over a threshold. We have observed that with a small amount of sparse elements, the model accuracy can be recovered significantly for the highly compressed CNN models. Based on this premise, we developed a novel method, called LPSD (Low-rank Plus Sparse Decomposition), that decomposes a CNN weight tensor into a combination of a low-rank and a sparse components, which can better maintain the accuracy for the high compression ratio. For a pretrained model, the network structure of each layer is split into two branches: one for low-rank part and one for sparse part. LPSD adapts the alternating approximation algorithm to minimize the global error and the local error alternatively. An exhausted search method with pruning is designed to search the optimal group number, ranks, and sparsity. Experimental results demonstrate that in most scenarios, LPSD achieves better accuracy compared to the state-of-the-art methods when the model is highly compressed.