OPTIMAL BOUNDS FOR AGGREGATION OF AFFINE ESTIMATORS

被引：13

作者：

Bellec, Pierre C. ^{[1
,2
,3
]}

机构：

[1] ENSAE, Malakoff, France

[2] Rutgers State Univ, New Brunswick, NJ USA

[3] Rutgers State Univ, Dept Stat & Biostat, 501 Hill Ctr,Busch Campus,110 Frelinghuysen Rd, Piscataway, NJ 08854 USA

来源：

ANNALS OF STATISTICS | 2018年 / 46卷 / 01期

关键词：

Affine estimator; aggregation; sequence model; sharp oracle inequality; concentration inequality; Hanson-Wright; INDEPENDENT RANDOM-VARIABLES; NONPARAMETRIC REGRESSION; LEAST-SQUARES; VARIANCE-ESTIMATION; TAIL PROBABILITIES; CONVEX AGGREGATION; DENSITY ESTIMATORS; SPARSE ESTIMATION; QUADRATIC-FORMS; MODEL SELECTION;

D O I：

10.1214/17-AOS1540

中图分类号：

O21 [概率论与数理统计]; C8 [统计学];

学科分类号：

020208 ; 070103 ; 0714 ;

摘要：

We study the problem of aggregation of estimators when the estimators are not independent of the data used for aggregation and no sample splitting is allowed. If the estimators are deterministic vectors, it is well known that the minimax rate of aggregation is of order log(M), where M is the number of estimators to aggregate. It is proved that for affine estimators, the minimax rate of aggregation is unchanged: it is possible to handle the linear dependence between the affine estimators and the data used for aggregation at no extra cost. The minimax rate is not impacted either by the variance of the affine estimators, or any other measure of their statistical complexity. The minimax rate is attained with a penalized procedure over the convex hull of the estimators, for a penalty that is inspired from the Q-aggregation procedure. The results follow from the interplay between the penalty, strong convexity and concentration.

引用

页码：30 / 59

页数：30

共 41 条

[1]

[Anonymous], 2013, Concentration Inequali-ties: A Nonasymptotic Theory of Independence, DOI DOI 10.1093/ACPROF:OSO/9780199535255.001.0001

[2]

ARLOT S., 2009, NIPS 2009 ADV NEURAL, P46

[3]

AUDIBERT J.-Y., 2007, PREPRINT

[4] Estimator selection in the Gaussian setting [J].

Baraud, Yannick ;

Giraud, Christophe ;

Huet, Sylvie .

ANNALES DE L INSTITUT HENRI POINCARE-PROBABILITES ET STATISTIQUES, 2014, 50 (03) :1092-1119

[5] Optimal exponential bounds for aggregation of density estimators [J].

Bellec, Pierre C. .

BERNOULLI, 2017, 23 (01) :219-248

[6] PIVOTAL ESTIMATION VIA SQUARE-ROOT LASSO IN NONPARAMETRIC REGRESSION [J].

Belloni, Alexandre ;

Chernozhukov, Victor ;

Wang, Lie .

ANNALS OF STATISTICS, 2014, 42 (02) :757-788

[7] Least squares after model selection in high-dimensional sparse models [J].

Belloni, Alexandre ;

Chernozhukov, Victor .

BERNOULLI, 2013, 19 (02) :521-547

[8]

BOYD S., 2009, CONVEX OPTIMIZATION

[9] ALL ADMISSIBLE LINEAR ESTIMATES OF MEAN VECTOR [J].

COHEN, A .

ANNALS OF MATHEMATICAL STATISTICS, 1966, 37 (02) :458-&

[10] Aggregation of affine estimators [J].

Dai, Dong ;

Rigollet, Philippe ;

Xia, Lucy ;

Zhang, Tong .

ELECTRONIC JOURNAL OF STATISTICS, 2014, 8 :302-327

← 1 2 3 4 5 →