Weak Convergence of the Regularization Path in Penalized M-Estimation

We consider a function defined as the pointwise minimization of a doubly index random process. We are interested in the weak convergence of the minimizer in the space of bounded functions. Such convergence results can be applied in the context of penalized M-estimation, that is, when the random process to minimize is expressed as a goodness-of-fit term plus a penalty term multiplied by a penalty weight. This weight is called the regularization parameter and the minimizing function the regularization path. The regularization path can be seen as a collection of estimators indexed by the regularization parameter. We obtain a consistency result and a central limit theorem for the regularization path in a functional sense. Various examples are provided, including the l1-regularization path for general linear models, the l1- or l2-regularization path of the least absolute deviation regression and the Akaike information criterion.