Tensor-Train Parameterization for Ultra Dimensionality Reduction

Dimensionality reduction is a conventional yet crucial field in machine learning. In dimensionality reduction, locality preserving projections (LPP) are a vital method designed to avoid the sensitivity to outliers based on data graph information. However, in terms of the extreme outliers, the performance of LPP is still largely undermined by them. For the case when the input data are matrices or tensors, LPP can only process them by flattening them into an extensively long vector and thus result in the loss of structural information. Furthermore, the assumption for LPP is that the dimension of data should be smaller than the number of instances. Therefore, for high-dimensional data analysis, LPP is not appropriate. In this case, the tensor-train decomposition comes to the stage and demonstrates the efficiency and effectiveness to capture these spatial relations. In consequence, a tensor-train parameterization for ultra dimensionality reduction (TTPUDR) is proposed in this paper, where the conventional LPP mapping is tensorized through tensor-trains and the objective function in the traditional LPP is substituted with the Frobenius norm instead of the squared Frobenius norm to enhance the robustness of the model. We also utilize the manifold optimization to assist the learning process of the model. We evaluate the performance of TTPUDR on classification problems versus the state-of-the-art methods and the past axiomatic methods and TTPUDR significantly outperforms them.