Extreme Learning Machine for Multilayer Perceptron

被引:1120
作者
Tang, Jiexiong [1 ]
Deng, Chenwei [1 ]
Huang, Guang-Bin [2 ]
机构
[1] Beijing Inst Technol, Sch Informat & Elect, Beijing 100081, Peoples R China
[2] Nanyang Technol Univ, Sch Elect & Elect Engn, Singapore 639798, Singapore
基金
中国国家自然科学基金;
关键词
Deep learning (DL); deep neural network (DNN); extreme learning machine (ELM); multilayer perceptron (MLP); random feature mapping; RECOGNITION; ALGORITHM;
D O I
10.1109/TNNLS.2015.2424995
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Extreme learning machine (ELM) is an emerging learning algorithm for the generalized single hidden layer feedforward neural networks, of which the hidden node parameters are randomly generated and the output weights are analytically computed. However, due to its shallow architecture, feature learning using ELM may not be effective for natural signals (e.g., images/videos), even with a large number of hidden nodes. To address this issue, in this paper, a new ELM-based hierarchical learning framework is proposed for multilayer perceptron. The proposed architecture is divided into two main components: 1) self-taught feature extraction followed by supervised feature classification and 2) they are bridged by random initialized hidden weights. The novelties of this paper are as follows: 1) unsupervised multilayer encoding is conducted for feature extraction, and an ELM-based sparse autoencoder is developed via l1 constraint. By doing so, it achieves more compact and meaningful feature representations than the original ELM; 2) by exploiting the advantages of ELM random feature mapping, the hierarchically encoded outputs are randomly projected before final decision making, which leads to a better generalization with faster learning speed; and 3) unlike the greedy layerwise training of deep learning (DL), the hidden layers of the proposed framework are trained in a forward manner. Once the previous layer is established, the weights of the current layer are fixed without fine-tuning. Therefore, it has much better learning efficiency than the DL. Extensive experiments on various widely used classification data sets show that the proposed algorithm achieves better and faster convergence than the existing state-of-the-art hierarchical learning methods. Furthermore, multiple applications in computer vision further confirm the generality and capability of the proposed learning scheme.
引用
收藏
页码:809 / 821
页数:13
相关论文
共 33 条
[21]  
Kim TK, 2007, LECT NOTES COMPUT SC, V4843, P335
[22]  
Kim TK, 2007, PROC CVPR IEEE, P1275
[23]  
LeCun Y, 2004, PROC CVPR IEEE, P97
[24]   Gradient-based learning applied to document recognition [J].
Lecun, Y ;
Bottou, L ;
Bengio, Y ;
Haffner, P .
PROCEEDINGS OF THE IEEE, 1998, 86 (11) :2278-2324
[25]   A fast and accurate online sequential learning algorithm for feedforward networks [J].
Liang, Nan-Ying ;
Huang, Guang-Bin ;
Saratchandran, P. ;
Sundararajan, N. .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 2006, 17 (06) :1411-1423
[26]   Tangent Bundles on Special Manifolds for Action Recognition [J].
Lui, Yui Man .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2012, 22 (06) :930-942
[27]   Human action recognition using extreme learning machine based on visual vocabularies [J].
Minhas, Rashid ;
Baradarani, Aryaz ;
Seifzadeh, Sepideh ;
Wu, Q. M. Jonathan .
NEUROCOMPUTING, 2010, 73 (10-12) :1906-1917
[28]   Human face recognition based on multidimensional PCA and extreme learning machine [J].
Mohammed, A. A. ;
Minhas, R. ;
Wu, Q. M. Jonathan ;
Sid-Ahmed, M. A. .
PATTERN RECOGNITION, 2011, 44 (10-11) :2588-2597
[29]   Leukocyte image segmentation by visual attention and extreme learning machine [J].
Pan, Chen ;
Park, Dong Sun ;
Yang, Yong ;
Yoo, Hyouck Min .
NEURAL COMPUTING & APPLICATIONS, 2012, 21 (06) :1217-1227
[30]   Incremental learning for robust visual tracking [J].
Ross, David A. ;
Lim, Jongwoo ;
Lin, Ruei-Sung ;
Yang, Ming-Hsuan .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 2008, 77 (1-3) :125-141