High-dimensional covariance estimation by minimizing l1-penalized log-determinant divergence

Given i.i.d. observations of a random vector X is an element of R-p, we study the problem of estimating both its covariance matrix Sigma*, and its inverse covariance or concentration matrix Theta* = (Sigma*)(-1). When X is multivariate Gaussian, the non-zero structure of Theta* is specified by the graph of an associated Gaussian Markov random field; and a popular estimator for such sparse Theta* is the l(1)-regularized Gaussian MLE. This estimator is sensible even for for non-Gaussian X, since it corresponds to minimizing an l(1)-penalized log-determinant Bregman divergence. We analyze its performance under high-dimensional scaling, in which the number of nodes in the graph p, the number of edges s, and the maximum node degree d, are allowed to grow as a function of the sample size n. In addition to the parameters (p, s, d), our analysis identifies other key quantities that control rates: (a) the l(infinity)-operator norm of the true covariance matrix Sigma*; and (b) the l(infinity)-operator norm of the sub-matrix Gamma(SS)*, where S indexes the graph edges, and Gamma* = (Theta*)(-1) circle times (Theta*)(-1); and (c) a mutual incoherence or irrepresentability measure on the matrix Gamma* and (d) the rate of decay 1/f(n, delta) on the probabilities {vertical bar(Sigma) over cap (n)(ij) - Sigma(ij)*vertical bar > delta}, where (Sigma) over cap (n) is the sample covariance based on n samples. Our first result establishes consistency of our estimate (Theta) over cap in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees d = o(root s). In our second result, we show that with probability converging to one, the estimate (Theta) over cap correctly specifies the zero pattern of the concentration matrix Theta*. We illustrate our theoretical results via simulations for various graphs and problem parameters, showing good correspondences between the theoretical predictions and behavior in simulations.