Emergent spatio-temporal multimodal learning using a developmental network

被引:8
作者
Wang, Dongshu [1 ]
Xin, Jianbin [1 ]
机构
[1] Zhengzhou Univ, Sch Elect Engn, 100,Sci Rd, Zhengzhou 450001, Henan, Peoples R China
基金
中国博士后科学基金; 中国国家自然科学基金;
关键词
Multimodal learning; Developmental network; Synapse maintenance; Skull-closed; ORIENTATION; OBJECT;
D O I
10.1007/s10489-018-1337-5
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Conventional machine learning needs humans to train each module with hand-handcrafted data and symbols manually, and the results of these methods are confined to particular tasks. To address this limitation, in this paper we design a multimodal autonomous learning architecture based on a developmental network for the audio and vision co-development. The developmental network is a biological inspired mechanism, which can make an agent to develop and integrate audition and vision simultaneously. Furthermore, synapse maintenance is introduced in the vision information learning to enhance the video recognition rate and neuron regenesis mechanism is implemented to enhance the network usage efficiency. In the experiments, a number of fundamental words are acquired and identified using the proposed learning methodology without any prior knowledge about the objects or the verbal questions before running. The experiments show that the proposed learning method can achieve significantly high recognition rates in comparison with the state-of-the-art method.
引用
收藏
页码:1306 / 1323
页数:18
相关论文
共 39 条
  • [1] Learning to Associate Auditory and Visual Stimuli: Behavioral and Neural Mechanisms
    Altieri, Nicholas
    Stevenson, Ryan A.
    Wallace, Mark T.
    Wenger, Michael J.
    [J]. BRAIN TOPOGRAPHY, 2015, 28 (03) : 479 - 493
  • [2] [Anonymous], COMPUTATIONAL COMPLE
  • [3] [Anonymous], 2008, P ICMI
  • [4] A deep architecture for audio-visual voice activity detection in the presence of transients
    Ariav, Ido
    Dov, David
    Cohen, Israel
    [J]. SIGNAL PROCESSING, 2018, 142 : 69 - 74
  • [5] Role of the right inferior parietal cortex in auditory selective attention: An rTMS study
    Bareham, Corinne A.
    Georgieva, Stanimira D.
    Kamke, Marc R.
    Lloyd, David
    Bekinschtein, Tristan A.
    Mattingley, Jason B.
    [J]. CORTEX, 2018, 99 : 30 - 38
  • [6] Bertenthal BI, 1984, CONTINUITIES DISCONT
  • [7] Rubber hands 'feel' touch that eyes see
    Botvinick, M
    Cohen, J
    [J]. NATURE, 1998, 391 (6669) : 756 - 756
  • [8] Detecting small group activities from multimodal observations
    Brdiczka, Oliver
    Maisonnasse, Jerome
    Reignier, Patrick
    Crowley, James L.
    [J]. APPLIED INTELLIGENCE, 2009, 30 (01) : 47 - 57
  • [9] Deep unsupervised network for multimodal perception, representation and classification
    Droniou, Alain
    Ivaldi, Serena
    Sigaud, Olivier
    [J]. ROBOTICS AND AUTONOMOUS SYSTEMS, 2015, 71 : 83 - 98
  • [10] Erhan D, 2010, J MACH LEARN RES, V11, P625