A new Sigma-Pi-Sigma neural network based on L1 and L2 regularization and applications

被引:2
作者
Jiao, Jianwei [1 ]
Su, Keqin [2 ]
机构
[1] Zhengzhou Inst Finance & Econ, Sch Civil Engn, Zhengzhou 450000, Peoples R China
[2] Henan Agr Univ, Coll Informat & Management Sci, Zhengzhou 450046, Peoples R China
来源
AIMS MATHEMATICS | 2024年 / 9卷 / 03期
关键词
a new Sigma-Pi-Sigma neural network; batch gradient method; regularization; convergence; EXTREME LEARNING-MACHINE; CONVERGENCE ANALYSIS; PRUNING ALGORITHM; PENALTY; BOUNDEDNESS;
D O I
10.3934/math.2024293
中图分类号
O29 [应用数学];
学科分类号
070104 ;
摘要
As one type of the important higher -order neural networks developed in the last decade, the Sigma -Pi -Sigma neural network has more powerful nonlinear mapping capabilities compared with other popular neural networks. This paper is concerned with a new Sigma -Pi -Sigma neural network based on a L1 and L2 regularization batch gradient method, and the numerical experiments for classification and regression problems prove that the proposed algorithm is effective and has better properties comparing with other classical penalization methods. The proposed model combines the sparse solution tendency of L1 norm and the high benefits in efficiency of the L2 norm, which can regulate the complexity of a network and prevent overfitting. Also, the numerical oscillation, induced by the non -differentiability of L1 plus L2 regularization at the origin, can be eliminated by a smoothing technique to approximate the objective function.
引用
收藏
页码:5995 / 6012
页数:18
相关论文
共 46 条
[1]   A LEMPEL-ZIV COMPLEXITY-BASED NEURAL NETWORK PRUNING ALGORITHM [J].
Ahmed, Sultan Uddin ;
Shahjahan, Md. ;
Murase, Kazuyuki .
INTERNATIONAL JOURNAL OF NEURAL SYSTEMS, 2011, 21 (05) :427-441
[2]   Pruning algorithms of neural networks - a comparative study [J].
Augasta, M. Gethsiyal ;
Kathirvalavakumar, T. .
OPEN COMPUTER SCIENCE, 2013, 3 (03) :105-115
[3]   A survey of artificial neural network training tools [J].
Baptista, Dario ;
Morgado-Dias, Fernando .
NEURAL COMPUTING & APPLICATIONS, 2013, 23 (3-4) :609-615
[4]  
[常象宇 Chang Xiangyu], 2010, [中国科学:数学, Scientia Sinica Mathematica], V40, P985
[5]   On different facets of regularization theory [J].
Chen, Z ;
Haykin, S .
NEURAL COMPUTATION, 2002, 14 (12) :2791-2846
[6]   Research on Adaptive Trajectory Tracking Algorithm for a Quadrotor Based on Backstepping and the Sigma-Pi Neural Network [J].
Chen, Zhiming ;
Niu, Kang ;
Li, Lei .
INTERNATIONAL JOURNAL OF AEROSPACE ENGINEERING, 2019, 2019
[7]   Elastic-net regularization in learning theory [J].
De Mol, Christine ;
De Vito, Ernesto ;
Rosasco, Lorenzo .
JOURNAL OF COMPLEXITY, 2009, 25 (02) :201-230
[8]  
Fahlman Scott., 1990, Cascade-Correlation
[9]   A Hybrid Higher Order Neural Classifier for handling classification problems [J].
Fallahnezhad, Mehdi ;
Moradi, Mohammad Hassan ;
Zaferanlouei, Salman .
EXPERT SYSTEMS WITH APPLICATIONS, 2011, 38 (01) :386-393
[10]  
Fan QW, 2024, IEEE T NEUR NET LEAR, V35, P18687, DOI 10.1109/TNNLS.2023.3319989