Optimization algorithms on subspaces: Revisiting missing data problem in low-rank matrix

Low-rank matrix approximation has applications in many fields, such as 3D reconstruction from an image sequence and 2D filter design. In this paper, one issue with low-rank matrix approximation is re-investigated: the missing data problem. Much effort was devoted to this problem, and the Wiberg algorithm or the damped Newton algorithm were recommended in previous studies. However, the Wiberg or damped Newton algorithms do not suit for large (especially "long") matrices, because one needs to solve a large linear system in every iteration. In this paper, we revitalize the usage of the Levenberg-Marquardt algorithm for solving the missing data problem, by utilizing the property that low-rank approximation is a minimization problem on subspaces. In two proposed implementations of the Levenberg-Marquardt algorithm, one only needs to solve a much smaller linear system in every iteration, especially for "long" matrices. Simulations and experiments on real data show the superiority of the proposed algorithms. Though the proposed algorithms achieve a high success rate in estimating the optimal solution by random initialization, as illustrated by real examples; it still remains an open issue how to properly do the initialization in a severe situation (that is, a large amount of data is missing and with high-level noise).