Feature Fusion Using Multiple Component Analysis

Canonical correlation analysis (CCA) and partial least squares (PLS) are always used as fusing two feature sets. How to extend them to fuse multiple features in a generalized way is still an unsolved problem. In this paper, we propose a novel feature fusion method called multiple component analysis (MCA). By constructing a higher-order tensor, all kinds of information are fused into the covariance tensor. Then orthogonal subspaces corresponding to each feature set are learned through tensor singular value decomposition (SVD), that couples dimension reduction and feature fusion together. Compared with multiple feature fusion by subspace learning (MFFSL), our method has the ability to represent fused data more efficiently and discriminatively in very few components. And it is shown that principle component analysis (PCA) and PLS are special cases of our method when there are only one set and two sets of features respectively. Extensive experiments on both handwritten numerals classification and face recognition demonstrate the effectiveness and robustness of the proposed method.