Bayesian estimation of Karhunen-Loeve expansions; A random subspace approach

被引:10
作者
Chowdhary, Kenny [1 ]
Najm, Habib N. [1 ]
机构
[1] Sandia Natl Labs, Livermore, CA USA
关键词
Karhunen-Loeve expansion; Principal Component Analysis; Uncertainty quantification; Bayesian inference; Matrix Bingham density; Gibbs sampling; Markov chain Monte Carlo;
D O I
10.1016/j.jcp.2016.02.056
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
One of the most widely-used procedures for dimensionality reduction of high dimensional data is Principal Component Analysis (PCA). More broadly, low-dimensional stochastic representation of random fields with finite variance is provided via the well known Karhunen-Loeve expansion (KLE). The KLE is analogous to a Fourier series expansion for a random process, where the goal is to find an orthogonal transformation for the data such that the projection of the data onto this orthogonal subspace is optimal in the L-2 sense, i.e., which minimizes the mean square error. In practice, this orthogonal transformation is determined by performing an SVD (Singular Value Decomposition) on the sample covariance matrix or on the data matrix itself. Sampling error is typically ignored when quantifying the principal components, or, equivalently, basis functions of the KLE. Furthermore, it is exacerbated when the sample size is much smaller than the dimension of the random field. In this paper, we introduce a Bayesian KLE procedure, allowing one to obtain a probabilistic model on the principal components, which can account for inaccuracies due to limited sample size. The probabilistic model is built via Bayesian inference, from which the posterior becomes the matrix Bingham density over the space of orthonormal matrices. We use a modified Gibbs sampling procedure to sample on this space and then build probabilistic Karhunen-Loeve expansions over random subspaces to obtain a set of low-dimensional surrogates of the stochastic process. We illustrate this probabilistic procedure with a finite dimensional stochastic process inspired by Brownian motion. (C) 2016 Elsevier Inc. All rights reserved.
引用
收藏
页码:280 / 293
页数:14
相关论文
共 16 条
[1]  
[Anonymous], 2002, Principal components analysis
[2]  
[Anonymous], 2010, SPECTRAL METHODS UNC
[3]   Estimation of covariance matrices based on hierarchical inverse-Wishart priors [J].
Bouriga, M. ;
Feron, O. .
JOURNAL OF STATISTICAL PLANNING AND INFERENCE, 2013, 143 (04) :795-808
[4]  
Cao G., 2011, IEEE T IMAGE PROCESS, V64, P27
[5]   EXPLAINING THE GIBBS SAMPLER [J].
CASELLA, G ;
GEORGE, EI .
AMERICAN STATISTICIAN, 1992, 46 (03) :167-174
[6]   UNDERSTANDING THE METROPOLIS-HASTINGS ALGORITHM [J].
CHIB, S ;
GREENBERG, E .
AMERICAN STATISTICIAN, 1995, 49 (04) :327-335
[7]  
Chikuse Chikuse Y. Y., Lecture Notes in Statistics)
[8]  
Gutierrez R., 1992, APPL STOCH MODEL D A, V8, P67
[9]   Simulation of the Matrix Bingham-von Mises-Fisher Distribution, With Applications to Multivariate and Relational Data [J].
Hoff, Peter D. .
JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS, 2009, 18 (02) :438-456
[10]  
JAIMEZ RG, 1987, TRAB ESTAD, V2, P81, DOI 10.1007/BF02863594