Adaptive Principal Component Analysis

Principal Component Analysis (PCA) is a powerful approach that can reduce the dimensionality of input data and increase the ease of data interpretation. One major obstacle for using the traditional PCA is its dependence on the squared l(2)-norm distance that is highly sensitive to outliers and noisy data. This significantly limits the applicability of the PCA when it is applied to large or high-dimensional datasets. In this paper, we review previous efforts of how to overcome this problem and further propose a novel adaptive PCA model. One broadly used hypothesis is that small losses of most data follow the Gaussian distribution and large losses of a few data follow the Laplacian distribution. With this recognition, we choose to use the adaptive loss function that integrates both l(1)-norm and squared l(2)-norm distances into a single loss function. To solve the resulted non-convex optimization problem, we derive an efficient iterative algorithm that guarantees that both the objective and solution sequences converge to global optimal solutions at a sub-linear convergence rate. The theoretical convergence analysis of our model is not trivial and comprehensively presented in our work. In comparison to other commonly used dimensionality reduction methods, our experiments on seven real-world datasets showed a great advantage of our model regardless of whether the data was noisy or not. We thus anticipate that our method will have broad usage for a wide range of real-world applications.