A NEW PERSPECTIVE ON LEAST SQUARES UNDER CONVEX CONSTRAINT

被引:52
作者
Chatterjee, Sourav [1 ]
机构
[1] Stanford Univ, Dept Stat, Stanford, CA 94305 USA
关键词
Least squares; maximum likelihood; convex constraint; empirical process; lasso; isotonic regression; denoising; REGRESSION SHRINKAGE; DANTZIG SELECTOR; LASSO; CONSISTENCY; RECOVERY; FREEDOM; REPRESENTATIONS; ASYMPTOTICS; PERSISTENCE; ESTIMATOR;
D O I
10.1214/14-AOS1254
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
Consider the problem of estimating the mean of a Gaussian random vector when the mean vector is assumed to be in a given convex set. The most natural solution is to take the Euclidean projection of the data vector on to this convex set; in other words, performing "least squares under a convex constraint." Many problems in modern statistics and statistical signal processing theory are special cases of this general situation. Examples include the lasso and other high-dimensional regression techniques, function estimation problems, matrix estimation and completion, shape-restricted regression, constrained denoising, linear inverse problems, etc. This paper presents three general results about this problem, namely, (a) an exact computation of the main term in the estimation error by relating it to expected maxima of Gaussian processes (existing results only give upper bounds), (b) a theorem showing that the least squares estimator is always admissible up to a universal constant in any problem of the above kind and (c) a counterexample showing that least squares estimator may not always be minimax rate-optimal. The result from part (a) is then used to compute the error of the least squares estimator in two examples of contemporary interest.
引用
收藏
页码:2340 / 2381
页数:42
相关论文
共 77 条
[1]  
[Anonymous], PREPRINT
[2]  
[Anonymous], 2001, CONCENTRATION MEASUR
[3]  
[Anonymous], PREPRINT
[4]  
[Anonymous], PREPRINT
[5]  
[Anonymous], 2009, ELEMENTS STAT LEARNI, DOI DOI 10.1007/978-0-387-84858-7
[6]   AN EMPIRICAL DISTRIBUTION FUNCTION FOR SAMPLING WITH INCOMPLETE INFORMATION [J].
AYER, M ;
BRUNK, HD ;
EWING, GM ;
REID, WT ;
SILVERMAN, E .
ANNALS OF MATHEMATICAL STATISTICS, 1955, 26 (04) :641-647
[7]   l1-regularized linear regression: persistence and oracle inequalities [J].
Bartlett, Peter L. ;
Mendelson, Shahar ;
Neeman, Joseph .
PROBABILITY THEORY AND RELATED FIELDS, 2012, 154 (1-2) :193-224
[8]   SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR [J].
Bickel, Peter J. ;
Ritov, Ya'acov ;
Tsybakov, Alexandre B. .
ANNALS OF STATISTICS, 2009, 37 (04) :1705-1732
[9]   APPROXIMATION IN METRIC-SPACES AND ESTIMATION THEORY [J].
BIRGE, L .
ZEITSCHRIFT FUR WAHRSCHEINLICHKEITSTHEORIE UND VERWANDTE GEBIETE, 1983, 65 (02) :181-237
[10]   RATES OF CONVERGENCE FOR MINIMUM CONTRAST ESTIMATORS [J].
BIRGE, L ;
MASSART, P .
PROBABILITY THEORY AND RELATED FIELDS, 1993, 97 (1-2) :113-150