PIANO: A fast parallel iterative algorithm for multinomial and sparse multinomial logistic regression

被引:2
作者
Jyothi, R. [1 ]
Babu, P. [1 ]
机构
[1] Indian Inst Technol, Ctr Appl Res Elect, Delhi, India
关键词
Multinomial logistic regression; Majorization minimization; Sparse; Parameter estimation; Regularization; Parallel algorithms;
D O I
10.1016/j.sigpro.2022.108459
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Multinomial Logistic Regression is a well-studied tool for classification and has been widely used in fields like image processing, computer vision and, bioinformatics, to name a few. Under a supervised classification scenario, a Multinomial Logistic Regression model learns a weight vector to differentiate between any two classes by optimizing over the likelihood objective. With the advent of big data, the inundation of data has resulted in large dimensional weight vector and has also given rise to a huge number of classes, which makes the classical methods applicable for model estimation not computationally viable. To handle this issue, we here propose a parallel iterative algorithm: Parallel Iterative Algorithm for MultiNomial LOgistic Regression ( PIANO ) which is based on the Majorization Minimization procedure, and can parallely update each element of the weight vectors. Further, we also show that PIANO can be easily extended to solve the Sparse Multinomial Logistic Regression problem -an extensively studied problem because of its attractive feature selection property. In particular, we work out the extension of PIANO to solve the Sparse Multinomial Logistic Regression problem with epsilon(1) and t 0 regularizations. We also prove that PIANO converges to a stationary point of the Multinomial and the Sparse Multinomial Logistic Regression problems. Simulations were conducted to compare PIANO with the existing methods, and it was found that the proposed algorithm performs better than the existing methods in terms of speed of convergence.(C) 2022 Elsevier B.V. All rights reserved.
引用
收藏
页数:14
相关论文
共 29 条
[1]  
[Anonymous], 2013, P 30 INT C MACHINE
[2]  
[Anonymous], 2020, Machine Learning: A Bayesian and Optimization Perspective
[3]   A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems [J].
Beck, Amir ;
Teboulle, Marc .
SIAM JOURNAL ON IMAGING SCIENCES, 2009, 2 (01) :183-202
[4]  
Bishop C.M., 2006, Information Science and Statistics
[5]   Distributed optimization and statistical learning via the alternating direction method of multipliers [J].
Boyd S. ;
Parikh N. ;
Chu E. ;
Peleato B. ;
Eckstein J. .
Foundations and Trends in Machine Learning, 2010, 3 (01) :1-122
[6]  
Boyd S., 2004, CONVEX OPTIMIZATION
[7]   Sparse Multinomial Logistic Regression via Approximate Message Passing [J].
Byrne, Evan ;
Schniter, Philip .
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2016, 64 (21) :5485-5498
[8]  
Dua D, 2017, UCI machine learning repository
[9]   Parallel Selective Algorithms for Nonconvex Big Data Optimization [J].
Facchinei, Francisco ;
Scutari, Gesualdo ;
Sagratella, Simone .
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2015, 63 (07) :1874-1889
[10]   Large-scale Bayesian logistic regression for text categorization [J].
Genkin, Alexander ;
Lewis, David D. ;
Madigan, David .
TECHNOMETRICS, 2007, 49 (03) :291-304