Inversion-free subsampling Newton's method for large sample logistic regression

被引:4
作者
Kirkby, J. Lars [1 ]
Nguyen, Dang H. [2 ]
Duy Nguyen [3 ]
Nguyen, Nhu N. [4 ]
机构
[1] Georgia Inst Technol, Sch Ind & Syst Engn, Atlanta, GA 30318 USA
[2] Univ Alabama, Dept Math, 345 Gordon Palmer Hall,Box 870350, Tuscaloosa, AL 35487 USA
[3] Marist Coll, Dept Math, 3399 North Rd, Poughkeepsie, NY 12601 USA
[4] Univ Connecticut, Dept Math, 341 Mansfield, Storrs, CT 06269 USA
基金
美国国家科学基金会;
关键词
Logistic regression; Massive data; Optimal subsampling; Newton's method; Gradient descent; Stochastic gradient descent; APPROXIMATION; CHALLENGES; MATRIX;
D O I
10.1007/s00362-021-01263-y
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
In this paper, we develop a subsampling Newton's method to efficiently approximate the maximum likelihood estimate in logistic regression, which is especially useful for large-sample problems. One distinct feature of our algorithm is that matrix inversion is not explicitly performed. We propose two algorithms which are used to construct iteratively a sequence of matrices which converge to the Hessian of the maximum likelihood function on the subsample. We provide numerical examples to show that the proposed method is efficient and robust.
引用
收藏
页码:943 / 963
页数:21
相关论文
共 30 条
[1]  
Bach F, 2013, ARXIV PREPRINT ARXIV
[2]   AN EFFICIENT STOCHASTIC NEWTON ALGORITHM FOR PARAMETER ESTIMATION IN LOGISTIC REGRESSIONS [J].
Bercu, Bernard ;
Codichon, Antoine ;
Portier, Bruno .
SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2020, 58 (01) :348-367
[3]  
Bishop C., 2006, Pattern Recognition and Machine Learning
[4]   OPTIMAL SURVEY SCHEMES FOR STOCHASTIC GRADIENT DESCENT WITH APPLICATIONS TO M-ESTIMATION [J].
Clemencon, Stephan ;
Bertail, Patrice ;
Chautru, Emilie ;
Papa, Guillaume .
ESAIM-PROBABILITY AND STATISTICS, 2019, 23 :310-337
[5]   PARAMETRIC LINK MODIFICATION OF BOTH TAILS IN BINARY REGRESSION [J].
CZADO, C .
STATISTICAL PAPERS, 1994, 35 (03) :189-201
[6]   Sampling Algorithms for l2 Regression and Applications [J].
Drineas, Petros ;
Mahoney, Michael W. ;
Muthukrishnan, S. .
PROCEEDINGS OF THE SEVENTHEENTH ANNUAL ACM-SIAM SYMPOSIUM ON DISCRETE ALGORITHMS, 2006, :1127-+
[7]  
Drineas P, 2012, J MACH LEARN RES, V13, P3475
[8]   Faster least squares approximation [J].
Drineas, Petros ;
Mahoney, Michael W. ;
Muthukrishnan, S. ;
Sarlos, Tamas .
NUMERISCHE MATHEMATIK, 2011, 117 (02) :219-249
[9]  
Dua D, 2017, UCI machine learning repository
[10]  
Duflo M., 2013, Random Iterative Models