A modified stochastic quasi-Newton algorithm for summing functions problem in machine learning

被引：2

作者：

Chen, Xiaoxuan ^{[1
]}

Feng, Haishan ^{[1
]}

机构：

[1] Guangxi Univ, Sch Math & Informat Sci, Ctr Appl Math Guangxi, Nanning, Guangxi, Peoples R China

来源：

JOURNAL OF APPLIED MATHEMATICS AND COMPUTING | 2023年 / 69卷 / 02期

基金：

中国国家自然科学基金;

关键词：

Machine learning; SQN method; Mini-batch setting; SUPERLINEAR CONVERGENCE;

D O I：

10.1007/s12190-022-01800-4

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

In this paper, a new stochastic quasi-Newton method (SQN) is proposed which has a different approximation of the Hessian inverse matrix H-k. The modified quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula which has a better approximation to Hessian matrix has not only the gradient variation but also the function value. Because of the special nature of the sum function, the mini-batch setting is built in the algorithm, and less compution cost can be guaranteed. The number of iterations reduce to at most O(epsilon (-1/1-beta)). The convergence analysis is established in this paper. The numerical experiments show that this algorithm is competitive to other algorithms.

引用

页码：1491 / 1506

页数：16

共 50 条

[11] Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations
Zhang, JZ
Xu, CX
JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2001, 137 (02) : 269 - 278
[12] A Proximal Stochastic Quasi-Newton Algorithm with Dynamical Sampling and Stochastic Line Search
Zhang, Mengxiang
Li, Shengjie
JOURNAL OF SCIENTIFIC COMPUTING, 2025, 102 (01)
[13] Solving the eigenvalue complementarity problem using a quasi-Newton algorithm
Arenas, Favian
Arias, Carlos
Perez, Rosana
REVISTA DE LA ACADEMIA COLOMBIANA DE CIENCIAS EXACTAS FISICAS Y NATURALES, 2022, 46 (179): : 325 - 338
[14] Improved quasi-Newton algorithm for the optimal coil design problem
Garda, Bartlomiej
2019 PROGRESS IN APPLIED ELECTRICAL ENGINEERING (PAEE), 2019,
[15] Quasi-Newton methods for stochastic optimization
Levy, MN
Trosset, MW
Kincaid, RR
ISUMA 2003: FOURTH INTERNATIONAL SYMPOSIUM ON UNCERTAINTY MODELING AND ANALYSIS, 2003, : 304 - 309
[16] Quasi-Newton adaptive algorithm for generalized symmetric eigenvalue problem
Natl Univ of Singapore, Singapore, Singapore
IEEE Trans Signal Process, 10 (2413-2422):
[17] Stochastic quasi-Newton molecular simulations
Chau, C. D.
Sevink, G. J. A.
Fraaije, J. G. E. M.
PHYSICAL REVIEW E, 2010, 82 (02):
[18] Faster Stochastic Quasi-Newton Methods
Zhang, Qingsong
Huang, Feihu
Deng, Cheng
Huang, Heng
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (09) : 4388 - 4397
[19] A quasi-Newton adaptive algorithm for generalized symmetric eigenvalue problem
Mathew, G
Reddy, VU
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 1996, 44 (10) : 2413 - 2422
[20] A QUASI-NEWTON ALGORITHM ON THE ORTHOGONAL MANIFOLD FOR NMF WITH TRANSFORM LEARNING
Ablin, Pierre
Fagot, Dylan
Wendt, Herwig
Gramfort, Alexandre
Fevotte, Cedric
2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2019, : 700 - 704

← 1 2 3 4 5 →