Sparse additive models

被引:354
作者
Ravikumar, Pradeep [2 ]
Lafferty, John
Liu, Han
Wasserman, Larry [1 ]
机构
[1] Carnegie Mellon Univ, Dept Stat, Pittsburgh, PA 15213 USA
[2] Univ Calif Berkeley, Berkeley, CA 94720 USA
基金
美国国家科学基金会;
关键词
Additive models; Lasso; Non-parametric regression; Sparsity; VARIABLE SELECTION; REGRESSION;
D O I
10.1111/j.1467-9868.2009.00718.x
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
We present a new class of methods for high dimensional non-parametric regression and classification called sparse additive models. Our methods combine ideas from sparse linear modelling and additive non-parametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. Sparse additive models are essentially a functional version of the grouped lasso of Yuan and Lin. They are also closely related to the COSSO model of Lin and Zhang but decouple smoothing and sparsity, enabling the use of arbitrary non-parametric smoothers. We give an analysis of the theoretical properties of sparse additive models and present empirical results on synthetic and real data, showing that they can be effective in fitting sparse non-parametric models in high dimensional data.
引用
收藏
页码:1009 / 1030
页数:22
相关论文
共 27 条
[1]   Regularization of wavelet approximations - Rejoinder [J].
Antoniadis, A ;
Fan, J .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2001, 96 (455) :964-967
[2]  
BUJA A, 1989, ANN STAT, V17, P453, DOI 10.1214/aos/1176347115
[3]   Sparsity oracle inequalities for the Lasso [J].
Bunea, Florentina ;
Tsybakov, Alexandre ;
Wegkamp, Marten .
ELECTRONIC JOURNAL OF STATISTICS, 2007, 1 :169-194
[4]   An iterative thresholding algorithm for linear inverse problems with a sparsity constraint [J].
Daubechies, I ;
Defrise, M ;
De Mol, C .
COMMUNICATIONS ON PURE AND APPLIED MATHEMATICS, 2004, 57 (11) :1413-1457
[5]  
DAUBECHIES I, 2007, ARXIV07064297 PRINC
[6]   Nonparametric inferences for additive models [J].
Fan, JQ ;
Jiang, JC .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2005, 100 (471) :890-907
[7]   Variable selection via nonconcave penalized likelihood and its oracle properties [J].
Fan, JQ ;
Li, RZ .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2001, 96 (456) :1348-1360
[8]   Persistence in high-dimensional linear predictor selection and the virtue of overparametrization [J].
Greenshtein, E ;
Ritov, Y .
BERNOULLI, 2004, 10 (06) :971-988
[9]  
Hastie Trevor, 1999, Generalized additive models
[10]  
Juditsky A, 2000, ANN STAT, V28, P681