Ensemble-based estimates of eigenvector error for empirical covariance matrices

Covariance matrices are fundamental to the analysis and forecast of economic, physical and biological systems. Although the eigenvalues {lambda(i)} and eigenvectors {u(i)} of a covariance matrix are central to such endeavours, in practice one must inevitably approximate the covariance matrix based on data with finite sample size n to obtain empirical eigenvalues {(lambda) over tilde (i)} and eigenvectors {(u) over tilde (i)}, and therefore understanding the error so introduced is of central importance. We analyse eigenvector error parallel to u(i) - u(i)parallel to(2) while leveraging the assumption that the true covariance matrix having size p is drawn from a matrix ensemble with known spectral properties-particularly, we assume the distribution of population eigenvalues weakly converges as p -> infinity to a spectral density rho(lambda) and that the spacing between population eigenvalues is similar to that for the Gaussian orthogonal ensemble. Our approach complements previous analyses of eigenvector error that require the full set of eigenvalues to be known, which can be computationally infeasible when p is large. To provide a scalable approach for uncertainty quantification of eigenvector error we consider a fixed eigenvalue lambda and approximate the distribution of the expected square error r = E [parallel to u(i) - (u) over tilde (i)parallel to(2)] across the matrix ensemble for all u(i) associated with lambda(i) = lambda. We find, for example, that for sufficiently large matrix size p and sample size n > p, the probability density of r scales as 1/nr(2). This power-law scaling implies that the eigenvector error is extremely heterogeneous-even if r is very small for most eigenvectors, it can be large for others with non-negligible probability. We support this and further results with numerical experiments.