Recursive ridge regression using second-order stochastic algorithms

被引：4

作者：

Godichon-Baggioni, Antoine ^{[1
]}

Lu, Wei ^{[2
]}

Portier, Bruno ^{[2
]}

机构：

[1] Sorbonne Univ, Lab Probabilites Stat & Modelisat, 4 Pl Jussieu, F-75005 Paris, France

[2] INSA Rouen Normandie, Lab Math INSA, 685 Ave Univ, F-76800 Saint Etienne Du Rouvray, France

来源：

COMPUTATIONAL STATISTICS & DATA ANALYSIS | 2024年 / 190卷

关键词：

Ridge regression; Stochastic optimization; Stochastic Newton algorithm; Recursive estimation; Machine learning;

D O I：

10.1016/j.csda.2023.107854

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

Recursive second-order stochastic algorithms are presented for solving ridge regression problems in the linear and binary logistic case. The proposed algorithms allow to update the estimates of ridge solution when the data arrive in continuous flow. Some guarantees on the almost sure behavior of the proposed algorithms are established. Numerical experiments on simulated and real-world data show the advantages of our algorithms compared to alternative methods.

引用

页数：16

共 33 条

[1]

An S, 2007, PROC CVPR IEEE, P1033

[2] COMBINING RIDGE AND PRINCIPAL COMPONENT REGRESSION - A MONEY DEMAND ILLUSTRATION [J].

BAYE, MR ;

PARKER, DF .

COMMUNICATIONS IN STATISTICS-THEORY AND METHODS, 1984, 13 (02) :197-205

[3] AN EFFICIENT STOCHASTIC NEWTON ALGORITHM FOR PARAMETER ESTIMATION IN LOGISTIC REGRESSIONS [J].

Bercu, Bernard ;

Codichon, Antoine ;

Portier, Bruno .

SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2020, 58 (01) :348-367

[4] Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables [J].

Blackard, JA ;

Dean, DJ .

COMPUTERS AND ELECTRONICS IN AGRICULTURE, 1999, 24 (03) :131-151

[5]

Boyer C, 2021, Arxiv, DOI arXiv:2011.09706

[6]

Cenac P., 2020, arXiv

[7]

Duflo M., 2013, RANDOM ITERATIVE MOD, V34

[8]

Franeois Kawala E. G., 2013, ACT C MOD AN RES APP, P16

[9] Regularization Paths for Generalized Linear Models via Coordinate Descent [J].

Friedman, Jerome ;

Hastie, Trevor ;

Tibshirani, Rob .

JOURNAL OF STATISTICAL SOFTWARE, 2010, 33 (01) :1-22

[10]

Gadat S., 2021, A stochastic Gauss-Newton algorithm for regularized semi-discrete optimal transport

← 1 2 3 4 →