Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection

被引:182
作者
Chen, Lisha [1 ]
Huang, Jianhua Z. [2 ]
机构
[1] Yale Univ, Dept Stat, New Haven, CT 06511 USA
[2] Texas A&M Univ, Dept Stat, College Stn, TX 77843 USA
基金
美国国家科学基金会;
关键词
Group lasso penalty; Low rank matrix approximation; Multivariate regression; Penalized least squares; Sparsity; Stiefel manifold; SACCHAROMYCES-CEREVISIAE; MULTIVARIATE REGRESSION; LINEAR-REGRESSION; LASSO; SHRINKAGE;
D O I
10.1080/01621459.2012.734178
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
The reduced-rank regression is an effective method in predicting multiple response variables from the same set of predictor variables. It reduces the dumber of model parameters and takes advantage of interrelations between. the response variables and hence improves predictive accuracy. We propose to select relevant variables for reduced-rank regression by using a sparsity-inducing penalty. We apply a group-lasso type penalty that treats each row of the matrix of the regression coefficients as a group and show that this penalty satisfies certain desirable invariance properties. We develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method. In particular, the manifold structure of the reduced-rank regression coefficient matrix is considered and studied in our theoretical analysis. In our simulation study and real data analysis, the new method is compared with several existing variable selection methods for multivariate regression and exhibits competitive performance in prediction and variable selection.
引用
收藏
页码:1533 / 1545
页数:13
相关论文
共 21 条
[1]  
Anderson TW, 1999, ANN STAT, V27, P1141
[2]  
[Anonymous], 2006, Journal of the Royal Statistical Society, Series B
[3]   Sparse partial least squares regression for simultaneous dimension reduction and variable selection [J].
Chun, Hyonho ;
Keles, Suenduez .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY, 2010, 72 :3-25
[4]   The geometry of algorithms with orthogonality constraints [J].
Edelman, A ;
Arias, TA ;
Smith, ST .
SIAM JOURNAL ON MATRIX ANALYSIS AND APPLICATIONS, 1998, 20 (02) :303-353
[5]   PATHWISE COORDINATE OPTIMIZATION [J].
Friedman, Jerome ;
Hastie, Trevor ;
Hoefling, Holger ;
Tibshirani, Robert .
ANNALS OF APPLIED STATISTICS, 2007, 1 (02) :302-332
[6]  
Gower J. C., 2004, Procrustes Problems
[7]  
Izenman A. J., 1975, Journal of Multivariate Analysis, V5, P248, DOI 10.1016/0047-259X(75)90042-1
[8]  
Izenman AJ, 2008, SPRINGER TEXTS STAT, P1, DOI 10.1007/978-0-387-78189-1_1
[9]   Transcriptional regulatory networks in Saccharomyces cerevisiae [J].
Lee, TI ;
Rinaldi, NJ ;
Robert, F ;
Odom, DT ;
Bar-Joseph, Z ;
Gerber, GK ;
Hannett, NM ;
Harbison, CT ;
Thompson, CM ;
Simon, I ;
Zeitlinger, J ;
Jennings, EG ;
Murray, HL ;
Gordon, DB ;
Ren, B ;
Wyrick, JJ ;
Tagne, JB ;
Volkert, TL ;
Fraenkel, E ;
Gifford, DK ;
Young, RA .
SCIENCE, 2002, 298 (5594) :799-804
[10]   REGULARIZED MULTIVARIATE REGRESSION FOR IDENTIFYING MASTER PREDICTORS WITH APPLICATION TO INTEGRATIVE GENOMICS STUDY OF BREAST CANCER [J].
Peng, Jie ;
Zhu, Ji ;
Bergamaschi, Anna ;
Han, Wonshik ;
Noh, Dong-Young ;
Pollack, Jonathan R. ;
Wang, Pei .
ANNALS OF APPLIED STATISTICS, 2010, 4 (01) :53-77