The Parallel Modification to the Levenberg-Marquardt Algorithm

被引：9

作者：

Bilski, Jaroslaw ^{[1
]}

Kowalczyk, Bartosz ^{[1
]}

Grzanek, Konrad ^{[2
,3
]}

机构：

[1] Czestochowa Tech Univ, Inst Computat Intelligence, Czestochowa, Poland

[2] Univ Social Sci, Informat Technol Inst, Lodz, Poland

[3] Clark Univ, Worcester, MA 01610 USA

来源：

ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, ICAISC 2018, PT I | 2018年 / 10841卷

关键词：

Feed-forward neural network; Parallel neural network training algorithm; Optimization problem; Levenberg-Marquardt algorithm; QR decomposition; Givens rotation;

D O I：

10.1007/978-3-319-91253-0_2

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The paper presents a parallel approach to the Levenberg-Marquardt algorithm (also called LM or LMA). The first section contains the mathematical basics of the classic LMA. Then the parallel modification to LMA is introduced. The classic Levenberg-Marquardt algorithm is sufficient for a training of small neural networks. For bigger networks the algorithm complexity becomes too big for the effective teaching. The main scope of this paper is to propose more complexity efficient approach to LMA by parallel computation. The proposed modification to LMA has been tested on a few function approximation problems and has been compared to the classic LMA. The paper concludes with the resolution that the parallel modification to LMA could significantly improve algorithm performance for bigger networks. Summary also contains a several proposals for the possible future work directions in the considered area.

引用

页码：15 / 24

页数：10

共 20 条

[1]

Bilski J, 2008, LECT NOTES ARTIF INT, V5097, P11, DOI 10.1007/978-3-540-69731-2_2

[2] Parallel Levenberg-Marquardt Algorithm Without Error Backpropagation [J].

Bilski, Jaroslaw ;

Wilamowski, Bogdan M. .

ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, ICAISC 2017, PT I, 2017, 10245 :25-39

[3] Parallel Architectures for Learning the RTRN and Elman Dynamic Neural Networks [J].

Bilski, Jaroslaw ;

Smolag, Jacek .

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2015, 26 (09) :2561-2570

[4] Parallel Approach to the Levenberg-Marquardt Learning Algorithm for Feedforward Neural Networks [J].

Bilski, Jaroslaw ;

Smolag, Jacek ;

Zurada, Jacek M. .

ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, PT I, 2015, 9119 :3-14

[5] A New Approach to Design of Control Systems Using Genetic Programming [J].

Cpalka, Krzysztof ;

Lapa, Krystian ;

Przybyl, Andrzej .

INFORMATION TECHNOLOGY AND CONTROL, 2015, 44 (04) :433-442

[6] TRAINING FEEDFORWARD NETWORKS WITH THE MARQUARDT ALGORITHM [J].

HAGAN, MT ;

MENHAJ, MB .

IEEE TRANSACTIONS ON NEURAL NETWORKS, 1994, 5 (06) :989-993

[7]

Khan NA, 2017, J ARTIF INTELL SOFT, V7, P215, DOI 10.1515/jaiscr-2017-0015

[8] On the Application of a Hybrid Genetic-Firework Algorithm for Controllers Structure and Parameters Selection [J].

Lapa, Krystian ;

Cpalka, Krzysztof .

INFORMATION SYSTEMS ARCHITECTURE AND TECHNOLOGY, ISAT 2015, PT I, 2016, 429 :111-123

[9] A New Interpretability Criteria for Neuro-Fuzzy Systems for Nonlinear Classification [J].

Lapa, Krystian ;

Cpalka, Krzysztof ;

Galushkin, Alexander I. .

ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, PT I, 2015, 9119 :448-468

[10]

Liu H, 2017, J ARTIF INTELL SOFT, V7, P111, DOI 10.1515/jaiscr-2017-0008

← 1 2 →