Sequential procedures for aggregating arbitrary estimators of a conditional mean

被引：18

作者：

Bunea, Florentina ^{[1
]}

Nobel, Andrew ^{[2
]}

机构：

[1] Florida State Univ, Dept Stat, Tallahassee, FL 32306 USA

[2] Univ N Carolina, Dept Stat & Operat Res, Chapel Hill, NC 27599 USA

来源：

IEEE TRANSACTIONS ON INFORMATION THEORY | 2008年 / 54卷 / 04期

基金：

美国国家科学基金会;

关键词：

aggregation; Bayesian model averaging; individual sequence; oracle inequality; prediction; regression;

D O I：

10.1109/TIT.2008.917657

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

In this correspondence, a sequential procedure for aggregating linear combinations of a finite family of regression estimates is described and analyzed. Particular attention is given to linear combinations having coefficients in the generalized simplex. The procedure is based on exponential weighting, and has a computationally tractable approximation. Analysis of the procedure is based in part on techniques from the sequential prediction of nonrandom sequences. Here these techniques are applied in a stochastic setting to obtain cumulative loss bounds for the aggregation procedure. From the cumulative loss bounds we derive an oracle inequality for the aggregate estimator for an unbounded response having a suitable moment-generating function. The inequality shows that the risk of the aggregate estimator is less than the risk of the best candidate linear combination in the generalized simplex, plus a complexity term that depends on the size of the coefficient set. The inequality readily yields convergence rates for aggregation over the unit simplex that are within logarithmic factors of known minimax bounds. Some preliminary results on model selection are also presented.

引用

页码：1725 / 1735

页数：11

共 37 条

[11]

CESABIANCHI N, 1999, P 12 ANN C COMP LEAR, P163

[12]

FOSTER DP, 1999, GAME ECON BEHAV, V29, P1084

[13]

Freund Y., 1996, Proceedings of the Ninth Annual Conference on Computational Learning Theory, P89, DOI 10.1145/238061.238072

[14]

Gelman A, 2003, BAYESIAN DATA ANAL

[15] A simple randomized algorithm for sequential prediction of ergodic time series [J].

Györfi, L ;

Lugosi, G ;

Morvai, G .

IEEE TRANSACTIONS ON INFORMATION THEORY, 1999, 45 (07) :2642-2650

[16]

Gyorfi L., 2001, MODELING UNCERTAINTY

[17] Sequential prediction of individual sequences under general loss functions [J].

Haussler, D ;

Kivinen, J ;

Warmuth, MK .

IEEE TRANSACTIONS ON INFORMATION THEORY, 1998, 44 (05) :1906-1925

[18]

Juditsky A, 2000, ANN STAT, V28, P681

[19] Recursive aggregation of estimators by the mirror descent algorithm with averaging [J].

Juditsky A.B. ;

Nazin A.V. ;

Tsybakov A.B. ;

Vayatis N. .

Problems of Information Transmission, 2005, 41 (4) :368-384

[20]

KIVINEN J, 1999, LECT NOTES COMPUTER, P153

← 1 2 3 4 →