INVESTIGATING ROBUSTNESS OF BIOLOGICAL VS. BACKPROP BASED LEARNING

被引:0
作者
Zhou, Yanpeng [1 ]
Wang, Maosen [1 ]
Gupta, Manas [2 ]
Ambikapathi, Arulmurugan [2 ,3 ]
Suganthan, Ponnuthurai Nagaratnam [1 ]
Ramasamy, Savitha [2 ,3 ]
机构
[1] Nanyang Technol Univ, Sch Elect & Elect Engn, Singapore, Singapore
[2] ASTAR, Inst Infocomm Res I2R, Singapore, Singapore
[3] ASTAR, Artificial Intelligence Analyt & Informat AI3, Singapore, Singapore
来源
2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP) | 2022年
关键词
Robustness; Hebbian learning algorithms; Representational learning; Biological plausible learning;
D O I
10.1109/ICASSP43922.2022.9747750
中图分类号
O42 [声学];
学科分类号
070206 ; 082403 ;
摘要
Robustness of learning algorithms remains an important problem to be solved from both the perspective of adversarial attacks and improving generalization. In this work, we investigate the robustness of biologically inspired Hebbian learning algorithm in depth. We find that Hebbian learning based algorithms outperform conventional learning algorithms like CNNs by a huge margin of upto 18% on the CIFAR-10 dataset under the addition of noise. We highlight that an important reason for this is the underlying representations that are being learnt by the learning algorithms. Specifically, we find that the Hebbian method learns the most robust representations compared to other methods that helps it to generalize better. We also conduct ablations on the Hebbian network and show-case that robustness of the model drops by upto 16% on the CIFAR-10 dataset if the representation capacity of the network is deteriorated. Hence, we find that the representations learnt play an important role in the resultant robustness of the models. We conduct experiments on multiple datasets and show that the results hold on all the datasets and at various noise levels.
引用
收藏
页码:3533 / 3537
页数:5
相关论文
共 23 条
[1]  
Allport D.A., 1985, Current perspectives in dysphasia, P32
[2]  
Amato G., 2019, International Conference on Image Analysis and Processing, V11751, DOI DOI 10.1007/978-3-030-30642-7_29
[3]  
Anderson Jr, 1976, SCIENCE, V12, P840
[4]  
FOMIN T, 1994, 1994 IEEE INTERNATIONAL CONFERENCE ON NEURAL NETWORKS, VOL 1-7, P731, DOI 10.1109/ICNN.1994.374267
[5]  
Goodfellow I, 2016, ADAPT COMPUT MACH LE, P1
[6]   HEBBNET: A SIMPLIFIED HEBBIAN LEARNING FRAMEWORK TO DO BIOLOGICALLY PLAUSIBLE LEARNING [J].
Gupta, Manas ;
Ambikapathi, ArulMurugan ;
Ramasamy, Savitha .
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021), 2021, :3115-3119
[7]  
Hansen M., 2018, SCI REP-UK, V8, P1, DOI DOI 10.1038/s41598-017-17765-5
[8]   Synergistic Gating of Electro-Iono-Photoactive 2D Chalcogenide Neuristors: Coexistence of Hebbian and Homeostatic Synaptic Metaplasticity [J].
John, Rohit Abraham ;
Liu, Fucai ;
Nguyen Anh Chien ;
Kulkarni, Mohit R. ;
Zhu, Chao ;
Fu, Qundong ;
Basu, Arindam ;
Liu, Zheng ;
Mathews, Nripan .
ADVANCED MATERIALS, 2018, 30 (25)
[9]  
Kermiche Noureddine, 2019, IEEE T NEURAL NETWOR, V31, P2118
[10]  
Klopf A. H., 1972, Brain Function and Adaptive Systems: A Heterostatic Theory