Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity

被引：92
作者：
Dalalyan, A. [1 ]
Tsybakov, A. B. [1 ,2 ]
机构：
[1] Univ Paris 06, LPMA, F-75252 Paris 05, France
[2] CREST, Stat Lab, F-92240 Malakoff, France
来源：
MACHINE LEARNING | 2008年 / 72卷 / 1-2期
关键词：
Aggregation; Nonparametric regression; Oracle inequalities; Sparsity;
D O I：
10.1007/s10994-008-5051-0
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
We study the problem of aggregation under the squared loss in the model of regression with deterministic design. We obtain sharp PAC-Bayesian risk bounds for aggregates defined via exponential weights, under general assumptions on the distribution of errors and on the functions to aggregate. We then apply these results to derive sparsity oracle inequalities.
引用
收藏
页码：39 / 61
页数：23
相关论文
共 42 条
[1]  
[Anonymous], THESIS U PARIS 6
[2]  
[Anonymous], 2004, MATH APPL
[3]   A randomized online learning algorithm for better variance control [J].
Audibert, Jean-Yves .
LEARNING THEORY, PROCEEDINGS, 2006, 4005 :392-407
[4]  
Bickel P.J., 2007, ANN STAT UNPUB
[5]  
BUNEA F, 2005, SEQUENTIAL PROCEDURE
[6]   Sparsity oracle inequalities for the Lasso [J].
Bunea, Florentina ;
Tsybakov, Alexandre ;
Wegkamp, Marten .
ELECTRONIC JOURNAL OF STATISTICS, 2007, 1 :169-194
[7]   Aggregation for gaussian regression [J].
Bunea, Florentina ;
Tsybakov, Alexandre B. ;
Wegkamp, Marten H. .
ANNALS OF STATISTICS, 2007, 35 (04) :1674-1697
[8]   Aggregation and sparsity via l1 penalized least squares [J].
Bunea, Florentina ;
Tsybakov, Alexandre B. ;
Wegkamp, Marten H. .
LEARNING THEORY, PROCEEDINGS, 2006, 4005 :379-391
[9]  
Candes E, 2007, ANN STAT, V35, P2313, DOI 10.1214/009053606000001523
[10]  
Catoni O., 2004, LECT NOTES MATH
12345