An enhanced learning algorithm with a particle filter-based gradient descent optimizer method

被引:0
作者
Patcharin Kamsing
Peerapong Torteeka
Soemsak Yooyen
机构
[1] King Mongkut’s Institute of Technology Ladkrabang,Department of Aeronautical Engineering, International Academy of Aviation Industry
[2] National Astronomical Research Institute of Thailand,undefined
来源
Neural Computing and Applications | 2020年 / 32卷
关键词
Gradient descent; Optimizer; Particle filter; Neural network; Deep learning;
D O I
暂无
中图分类号
学科分类号
摘要
This experiment integrates a particle filter concept with a gradient descent optimizer to reduce loss during iteration and obtains a particle filter-based gradient descent (PF-GD) optimizer that can determine the global minimum with excellent performance. Four functions are applied to test optimizer deployment to verify the PF-GD method. Additionally, the Modified National Institute of Standards and Technology (MNIST) database is used to test the PF-GD method by implementing a logistic regression learning algorithm. The experimental results obtained with the four functions illustrate that the PF-GD method performs much better than the conventional gradient descent optimizer, although it has some parameters that must be set before modeling. The results of implementing the MNIST dataset demonstrate that the cross-entropy of the PF-GD method exhibits a smaller decrease than that of the conventional gradient descent optimizer, resulting in higher accuracy of the PF-GD method. The PF-GD method provides the best accuracy for the training model, 97.00%, and the accuracy of evaluating the model with the test dataset is 90.37%, which is higher than the accuracy of 90.08% obtained with the conventional gradient descent optimizer.
引用
收藏
页码:12789 / 12800
页数:11
相关论文
共 65 条
[1]  
Naderpour H(2015)Application of soft computing to reinforced concrete beams strengthened with fibre reinforced polymers: a state-of-the-art review Comput Tech Civ Struct Eng 38 305-323
[2]  
Mirrashid M(2013)Soft computing applications in customer segmentation: state-of-art review and critique Expert Syst Appl 40 6491-6507
[3]  
Hiziroglu A(2019)Adaptive neuro-fuzzy inference modelling and sensitivity analysis for capacity estimation of fiber reinforced polymer-strengthened circular reinforced concrete columns Expert Syst 36 e12410-75
[4]  
Naderpour H(2019)An innovative approach for bond strength modeling in FRP strip-to-concrete joints using adaptive neuro–fuzzy inference system Eng Comput 24 04019006-59
[5]  
Nagai K(2019)Shear failure capacity prediction of concrete beam–column joints in terms of ANFIS and GMDH Pract Period Struct Des Constr 74 58-39
[6]  
Haji M(2017)Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: methods and applications Neurosci Biobehav Rev 230 48-464
[7]  
Mirrashid M(2017)A fast and memory saved GPU acceleration algorithm of convolutional neural networks for target detection Neurocomputing 18 35-46
[8]  
Naderpour H(2017)Radio frequency interference mitigation using deep convolutional neural networks Astron Comput 70 423-203
[9]  
Mirrashid M(2018)Soft computing based object detection and tracking approaches: state-of-the-art survey Appl Soft Comput 273 37-2648
[10]  
Nagai K(2018)A joint residual network with paired ReLUs activation for image super-resolution Neurocomputing 58 190-103