Improving Parametric PCA Using KL-divergence Between Gaussian-Markov Random Field Models

Parametric PCA is a dimensionality reduction based metric learning method that uses the Bhattacharrya and Hellinger distances between Gaussian distributions estimated from local patches of the KNN graph to build the parametric covariance matrix. Later on, PCA-KL, an entropic PCA version using the symmetrized KL-divergence (relative entropy) was proposed. In this paper, we extend this method by replacing the Gaussian distribution by a Gaussian-Markov random field model. The main advantage is the incorporation of the spatial dependence by means of the inverse temperature parameter. A closed form expression for the KL-divergence is derived, allowing fast computation and avoiding numerical simulations. Results with several real datasets show that the proposed method is capable of improving the average classification performance in comparison to PCA-KL and some state-of-the-art manifold learning algorithms, such as t-SNE and UMAP.