A Novel Sparse Penalty for Singular Value Decomposition

Singular value decomposition (SVD) is a tool widely used in data denoising, matrix approximation, recommendation system, text mining and computer vision. A majority of applications prefer sparse singular vectors to capture inherent structures and patterns of the input data so that the results are interpretable. We present a novel penalty for SVD to achieve sparsity. Comparing with the traditional penalties, the proposed penalty is scale, dimensional insensitive and bounded between 0 and 1, which are in favor of controlling sparsity. Regulated by the penalty, we provide an efficient algorithm to project a vector onto a given sparse level in O(n) expected time. The efficient projection algorithm serve as a drudge for sparse SVD (SSVD). In experiments, SSVD is efficient and could capture the latent structures and patterns of the input data.