A dynamic ensemble learning algorithm for neural networks

被引:378
|
作者
Alam, Kazi Md Rokibul [1 ]
Siddique, Nazmul [2 ]
Adeli, Hojjat [3 ,4 ,5 ]
机构
[1] Khulna Univ Engn & Technol, Dept Comp Sci & Engn, Khulna 9203, Bangladesh
[2] Ulster Univ, Sch Comp Engn & Intelligent Syst, Derry BT48 7JL, Londonderry, North Ireland
[3] Ohio State Univ, Dept Neurosci, Columbus, OH 43210 USA
[4] Ohio State Univ, Dept Neurol, Columbus, OH 43210 USA
[5] Ohio State Univ, Dept Biomed Informat, Columbus, OH 43210 USA
关键词
Neural network ensemble; Backpropagation algorithm; Negative correlation learning; Constructive algorithms; Pruning algorithms; SELECTION; DIVERSITY; CLASSIFICATION; CLASSIFIERS; COMBINATION; DESIGN;
D O I
10.1007/s00521-019-04359-7
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper presents a novel dynamic ensemble learning (DEL) algorithm for designing ensemble of neural networks (NNs). DEL algorithm determines the size of ensemble, the number of individual NNs employing a constructive strategy, the number of hidden nodes of individual NNs employing a constructive-pruning strategy, and different training samples for individual NN's learning. For diversity, negative correlation learning has been introduced and also variation of training samples has been made for individual NNs that provide better learning from the whole training samples. The major benefits of the proposed DEL compared to existing ensemble algorithms are (1) automatic design of ensemble; (2) maintaining accuracy and diversity of NNs at the same time; and (3) minimum number of parameters to be defined by user. DEL algorithm is applied to a set of real-world classification problems such as the cancer, diabetes, heart disease, thyroid, credit card, glass, gene, horse, letter recognition, mushroom, and soybean datasets. It has been confirmed by experimental results that DEL produces dynamic NN ensembles of appropriate architecture and diversity that demonstrate good generalization ability.
引用
收藏
页码:8675 / 8690
页数:16
相关论文
共 50 条
  • [41] Global learning of neural networks by using hybrid optimization algorithm
    Cho, Yong-Hyun
    Hong, Seong-Jun
    PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS AND KNOWLEDGE ENGINEERING (ISKE 2007), 2007,
  • [42] A backpropagation learning algorithm with graph regularization for feedforward neural networks
    Fan, Yetian
    Yang, Wenyu
    INFORMATION SCIENCES, 2022, 607 : 263 - 277
  • [43] A novel decorrelated neural network ensemble algorithm for face recognition
    Dai, Kankan
    Zhao, Jianwei
    Cao, Feilong
    KNOWLEDGE-BASED SYSTEMS, 2015, 89 : 541 - 552
  • [44] Ensemble of evolving neural networks in classification
    Sohn, SH
    Dagli, CHH
    NEURAL PROCESSING LETTERS, 2004, 19 (03) : 191 - 203
  • [45] Ensemble of Evolving Neural Networks in Classification
    Sunghwan Sohn
    Cihan H. Dagli
    Neural Processing Letters, 2004, 19 : 191 - 203
  • [46] Ensemble of convolutional neural networks based on an evolutionary algorithm applied to an industrial welding process
    Cruz, Yarens J.
    Rivas, Marcelino
    Quiza, Ramon
    Villalonga, Alberto
    Haber, Rodolfo E.
    Beruvides, Gerardo
    COMPUTERS IN INDUSTRY, 2021, 133
  • [47] A novel density-based ensemble learning algorithm with application to protein structural classification
    Homayouni, Haleh
    Mansoori, Eghbal G.
    INTELLIGENT DATA ANALYSIS, 2017, 21 (01) : 167 - 179
  • [48] Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis
    Poria, Soujanya
    Peng, Haiyun
    Hussain, Amir
    Howard, Newton
    Cambria, Erik
    NEUROCOMPUTING, 2017, 261 : 217 - 230
  • [49] Error-Correcting Output Codes with Ensemble Diversity for Robust Learning in Neural Networks
    Song, Yang
    Kang, Qiyu
    Tay, Wee Peng
    THIRTY-FIFTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THIRTY-THIRD CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE AND THE ELEVENTH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2021, 35 : 9722 - 9729