Optimizing the Parameters of Long Short-Term Memory Networks Using the Bees Algorithm

被引:7
|
作者
Alamri, Nawaf Mohammad H. [1 ]
Packianather, Michael [1 ]
Bigot, Samuel [1 ]
机构
[1] Cardiff Univ, Sch Engn, Queens Bldg,14-17 Parade, Cardiff CF24 3AA, Wales
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 04期
关键词
Deep Learning (DL); Long Short-Term Memory (LSTM); Convolutional Neural Network (CNN); Bees Algorithm (BA); Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM); NEURAL-NETWORKS;
D O I
10.3390/app13042536
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Improving the performance of Deep Learning (DL) algorithms is a challenging problem. However, DL is applied to different types of Deep Neural Networks, and Long Short-Term Memory (LSTM) is one of them that deals with time series or sequential data. This paper attempts to overcome this problem by optimizing LSTM parameters using the Bees Algorithm (BA), which is a nature-inspired algorithm that mimics the foraging behavior of honey bees. In particular, it was used to optimize the adjustment factors of the learning rate in the forget, input, and output gates, in addition to cell candidate, in both forward and backward sides. Furthermore, the BA was used to optimize the learning rate factor in the fully connected layer. In this study, artificial porosity images were used for testing the algorithms; since the input data were images, a Convolutional Neural Network (CNN) was added in order to extract the features in the images to feed into the LSTM for predicting the percentage of porosity in the sequential layers of artificial porosity images that mimic real CT scan images of products manufactured by the Selective Laser Melting (SLM) process. Applying a Convolutional Neural Network Long Short-Term Memory (CNN-LSTM) yielded a porosity prediction accuracy of 93.17%. Although using Bayesian Optimization (BO) to optimize the LSTM parameters mentioned previously did not improve the performance of the LSTM, as the prediction accuracy was 93%, adding the BA to optimize the same LSTM parameters did improve its performance in predicting the porosity, with an accuracy of 95.17% where a hybrid Bees Algorithm Convolutional Neural Network Long Short-Term Memory (BA-CNN-LSTM) was used. Furthermore, the hybrid BA-CNN-LSTM algorithm was capable of dealing with classification problems as well. This was shown by applying it to Electrocardiogram (ECG) benchmark images, which improved the test set classification accuracy, which was 92.50% for the CNN-LSTM algorithm and 95% for both the BO-CNN-LSTM and BA-CNN-LSTM algorithms. In addition, the turbofan engine degradation simulation numerical dataset was used to predict the Remaining Useful Life (RUL) of the engines using the LSTM network. A CNN was not needed in this case, as there was no feature extraction for the images. However, adding the BA to optimize the LSTM parameters improved the prediction accuracy in the testing set for the LSTM and BO-LSTM, which increased from 74% to 77% for the hybrid BA-LSTM algorithm.
引用
收藏
页数:27
相关论文
共 50 条
  • [1] Short-Term Traffic Prediction Using Long Short-Term Memory Neural Networks
    Abbas, Zainab
    Al-Shishtawy, Ahmad
    Girdzijauskas, Sarunas
    Vlassov, Vladimir
    2018 IEEE INTERNATIONAL CONGRESS ON BIG DATA (IEEE BIGDATA CONGRESS), 2018, : 57 - 65
  • [2] An accident diagnosis algorithm using long short-term memory
    Yang, Jaemin
    Kim, Jonghyun
    NUCLEAR ENGINEERING AND TECHNOLOGY, 2018, 50 (04) : 582 - 588
  • [3] Reliability Estimation Using Long Short-Term Memory Networks
    Davila-Frias, Alex
    Khumprom, Phattara
    Yadav, Om Prakash
    2023 ANNUAL RELIABILITY AND MAINTAINABILITY SYMPOSIUM, RAMS, 2023,
  • [4] Classification of HRV using Long Short-Term Memory Networks
    Leite, Argentina
    Silva, Maria Eduarda
    Rocha, Ana Paula
    2020 11TH CONFERENCE OF THE EUROPEAN STUDY GROUP ON CARDIOVASCULAR OSCILLATIONS (ESGCO): COMPUTATION AND MODELLING IN PHYSIOLOGY NEW CHALLENGES AND OPPORTUNITIES, 2020,
  • [5] A Novel Word Spotting Algorithm Using Bidirectional Long Short-Term Memory Neural Networks
    Frinken, Volkmar
    Fischer, Andreas
    Bunke, Horst
    ARTIFICIAL NEURAL NETWORKS IN PATTERN RECOGNITION, PROCEEDINGS, 2010, 5998 : 185 - 196
  • [6] On the Initialization of Long Short-Term Memory Networks
    Ghazi, Mostafa Mehdipour
    Nielsen, Mads
    Pai, Akshay
    Modat, Marc
    Cardoso, M. Jorge
    Ourselin, Sebastien
    Sorensen, Lauge
    NEURAL INFORMATION PROCESSING (ICONIP 2019), PT I, 2019, 11953 : 275 - 286
  • [7] Evolving Long Short-Term Memory Networks
    Neto, Vicente Coelho Lobo
    Passos, Leandro Aparecido
    Papa, Joao Paulo
    COMPUTATIONAL SCIENCE - ICCS 2020, PT II, 2020, 12138 : 337 - 350
  • [8] Deepfake Detection using Capsule Networks and Long Short-Term Memory Networks
    Mehra, Akul
    Spreeuwers, Luuk
    Strisciuglio, Nicola
    VISAPP: PROCEEDINGS OF THE 16TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS - VOL. 4: VISAPP, 2021, : 407 - 414
  • [9] Short-Term Load Forecasting using Long Short Term Memory Optimized by Genetic Algorithm
    Zulfiqar, Muhammad
    Rasheed, Muhammad Babar
    2022 IEEE SUSTAINABLE POWER AND ENERGY CONFERENCE (ISPEC), 2022,
  • [10] Subclinical tremor differentiation using long short-term memory networks
    Nanayakkara, Gerard Ruchin Randil
    Chan, Ping Yi
    PHYSICAL AND ENGINEERING SCIENCES IN MEDICINE, 2025,