Neuroinspired unsupervised learning and pruning with subquantum CBRAM arrays

被引:73
作者
Shi, Yuhan [1 ]
Nguyen, Leon [1 ]
Oh, Sangheon [1 ]
Liu, Xin [1 ]
Koushan, Foroozan [2 ]
Jameson, John R. [2 ]
Kuzum, Duygu [1 ]
机构
[1] Univ Calif San Diego, Dept Elect & Comp Engn, La Jolla, CA 92093 USA
[2] Adesto Technol Corp, 3600 Peterson Way, Santa Clara, CA 95054 USA
基金
美国国家科学基金会;
关键词
DEEP NEURAL-NETWORKS;
D O I
10.1038/s41467-018-07682-0
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Resistive RAM crossbar arrays offer an attractive solution to minimize off-chip data transfer and parallelize on-chip computations for neural networks. Here, we report a hardware/ software co-design approach based on low energy subquantum conductive bridging RAM (CBRAM (R)) devices and a network pruning technique to reduce network level energy consumption. First, we demonstrate low energy subquantum CBRAM devices exhibiting gradual switching characteristics important for implementing weight updates in hardware during unsupervised learning. Then we develop a network pruning algorithm that can be employed during training, different from previous network pruning approaches applied for inference only. Using a 512 kbit subquantum CBRAM array, we experimentally demonstrate high recognition accuracy on the MNIST dataset for digital implementation of unsupervised learning. Our hardware/software co-design approach can pave the way towards resistive memory based neuro-inspired systems that can autonomously learn and process information in power-limited settings.
引用
收藏
页数:11
相关论文
共 57 条
[1]   Equivalent-accuracy accelerated neural-network training using analogue memory [J].
Ambrogio, Stefano ;
Narayanan, Pritish ;
Tsai, Hsinyu ;
Shelby, Robert M. ;
Boybat, Irem ;
di Nolfo, Carmelo ;
Sidler, Severin ;
Giordano, Massimo ;
Bodini, Martina ;
Farinha, Nathan C. P. ;
Killeen, Benjamin ;
Cheng, Christina ;
Jaoudi, Yassine ;
Burr, Geoffrey W. .
NATURE, 2018, 558 (7708) :60-+
[2]  
BAYAT FM, 2018, NAT COMMUN, V9
[3]   Neuromorphic computing with multi-memristive synapses [J].
Boybat, Irem ;
Le Gallo, Manuel ;
Nandakumar, S. R. ;
Moraitis, Timoleon ;
Parnell, Thomas ;
Tuma, Tomas ;
Rajendran, Bipin ;
Leblebici, Yusuf ;
Sebastian, Abu ;
Eleftheriou, Evangelos .
NATURE COMMUNICATIONS, 2018, 9
[4]   DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving [J].
Chen, Chenyi ;
Seff, Ari ;
Kornhauser, Alain ;
Xiao, Jianxiong .
2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015, :2722-2730
[5]  
Chen P.-Y, 2017, NEUROINSPIRED COMPUT, V12, P233
[6]   NeuroSim: A Circuit-Level Macro Model for Benchmarking Neuro-Inspired Architectures in Online Learning [J].
Chen, Pai-Yu ;
Peng, Xiaochen ;
Yu, Shimeng .
IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, 2018, 37 (12) :3067-3080
[7]  
Chen PY, 2015, ICCAD-IEEE ACM INT, P194, DOI 10.1109/ICCAD.2015.7372570
[8]   Data Clustering using Memristor Networks [J].
Choi, Shinhyun ;
Sheridan, Patrick ;
Lu, Wei D. .
SCIENTIFIC REPORTS, 2015, 5
[9]   Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks [J].
Ciresan, Dan C. ;
Giusti, Alessandro ;
Gambardella, Luca M. ;
Schmidhuber, Juergen .
MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION - MICCAI 2013, PT II, 2013, 8150 :411-418
[10]  
Collobert R., 2008, P 25 ICML, P160, DOI [DOI 10.1145/1390156.1390177, 10.1145/1390156.1390177]