A structural developmental neural network with information saturation for continual unsupervised learning

被引:10
作者
Ding, Zhiyong [1 ]
Xie, Haibin [1 ,2 ]
Li, Peng [1 ]
Xu, Xin [1 ]
机构
[1] Natl Univ Def Technol, Coll Intelligence Sci & Technol, Changsha, Peoples R China
[2] Natl Univ Def Technol, Coll Intelligence Sci & Technol, Changsha 410073, Peoples R China
基金
中国国家自然科学基金;
关键词
neural network; pattern classification; unsupervised learning; SELF-ORGANIZING NETWORK; PATTERN-RECOGNITION;
D O I
10.1049/cit2.12169
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, we propose a structural developmental neural network to address the plasticity-stability dilemma, computational inefficiency, and lack of prior knowledge in continual unsupervised learning. This model uses competitive learning rules and dynamic neurons with information saturation to achieve parameter adjustment and adaptive structure development. Dynamic neurons adjust the information saturation after winning the competition and use this parameter to modulate the neuron parameter adjustment and the division timing. By dividing to generate new neurons, the network not only keeps sensitive to novel features but also can subdivide classes learnt repeatedly. The dynamic neurons with information saturation and division mechanism can simulate the long short-term memory of the human brain, which enables the network to continually learn new samples while maintaining the previous learning results. The parent-child relationship between neurons arising from neuronal division enables the network to simulate the human cognitive process that gradually refines the perception of objects. By setting the clustering layer parameter, users can choose the desired degree of class subdivision. Experimental results on artificial and real-world datasets demonstrate that the proposed model is feasible for unsupervised learning tasks in instance increment and class increment scenarios and outperforms prior structural developmental neural networks.
引用
收藏
页码:780 / 795
页数:16
相关论文
共 37 条
[1]  
Angelo C., 2015, DEV ROBOTICS BABIES
[2]   Multimodal Machine Learning: A Survey and Taxonomy [J].
Baltrusaitis, Tadas ;
Ahuja, Chaitanya ;
Morency, Louis-Philippe .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2019, 41 (02) :423-443
[3]   A Common Embryonic Origin of Stem Cells Drives Developmental and Adult Neurogenesis [J].
Berg, Daniel A. ;
Su, Yijing ;
Jimenez-Cyrus, Dennisse ;
Patel, Aneek ;
Huang, Nancy ;
Morizet, David ;
Lee, Stephanie ;
Shah, Reeti ;
Ringeling, Francisca Rojas ;
Jain, Rajan ;
Epstein, Jonathan A. ;
Wu, Qing-Feng ;
Canzar, Stefan ;
Ming, Guo-Li ;
Song, Hongjun ;
Bond, Allison M. .
CELL, 2019, 177 (03) :654-+
[4]   THE ART OF ADAPTIVE PATTERN-RECOGNITION BY A SELF-ORGANIZING NEURAL NETWORK [J].
CARPENTER, GA ;
GROSSBERG, S .
COMPUTER, 1988, 21 (03) :77-88
[5]   ART-3 - HIERARCHICAL SEARCH USING CHEMICAL TRANSMITTERS IN SELF-ORGANIZING PATTERN-RECOGNITION ARCHITECTURES [J].
CARPENTER, GA ;
GROSSBERG, S .
NEURAL NETWORKS, 1990, 3 (02) :129-152
[6]   ART-2 - SELF-ORGANIZATION OF STABLE CATEGORY RECOGNITION CODES FOR ANALOG INPUT PATTERNS [J].
CARPENTER, GA ;
GROSSBERG, S .
APPLIED OPTICS, 1987, 26 (23) :4919-4930
[7]   A Continual Learning Survey: Defying Forgetting in Classification Tasks [J].
De Lange, Matthias ;
Aljundi, Rahaf ;
Masana, Marc ;
Parisot, Sarah ;
Jia, Xu ;
Leonardis, Ales ;
Slabaugh, Greg ;
Tuytelaars, Tinne .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (07) :3366-3385
[8]   Learning in Nonstationary Environments: A Survey [J].
Ditzler, Gregory ;
Roveri, Manuel ;
Alippi, Cesare ;
Polikar, Robi .
IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE, 2015, 10 (04) :12-25
[9]   Emotion Recognition from Body Expressions with a Neural Network Architecture [J].
Elfaramawy, Nourhan ;
Barros, Pablo ;
Parisi, German I. ;
Wermter, Stefan .
PROCEEDINGS OF THE 5TH INTERNATIONAL CONFERENCE ON HUMAN AGENT INTERACTION (HAI'17), 2017, :143-149
[10]   A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis [J].
Fahad, Adil ;
Alshatri, Najlaa ;
Tari, Zahir ;
Alamri, Abdullah ;
Khalil, Ibrahim ;
Zomaya, Albert Y. ;
Foufou, Sebti ;
Bouras, Abdelaziz .
IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTING, 2014, 2 (03) :267-279