Experimental Demonstration of Supervised Learning in Spiking Neural Networks with Phase-Change Memory Synapses

被引:64
作者
Nandakumar, S. R. [1 ,2 ]
Boybat, Irem [1 ,3 ]
Le Gallo, Manuel [1 ]
Eleftheriou, Evangelos [1 ]
Sebastian, Abu [1 ]
Rajendran, Bipin [1 ,4 ]
机构
[1] IBM Res Zurich, CH-8803 Ruschlikon, Switzerland
[2] New Jersey Inst Technol, Newark, NJ 07102 USA
[3] Ecole Polytech Fed Lausanne EPFL, CH-1015 Lausanne, Switzerland
[4] Kings Coll London, London WC2R 2LS, England
基金
美国国家科学基金会; 欧洲研究理事会;
关键词
PRECISION; TRAINS; IMPACT; POWER;
D O I
10.1038/s41598-020-64878-5
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Spiking neural networks (SNN) are computational models inspired by the brain's ability to naturally encode and process information in the time domain. The added temporal dimension is believed to render them more computationally efficient than the conventional artificial neural networks, though their full computational capabilities are yet to be explored. Recently, in-memory computing architectures based on non-volatile memory crossbar arrays have shown great promise to implement parallel computations in artificial and spiking neural networks. In this work, we evaluate the feasibility to realize high-performance event-driven in-situ supervised learning systems using nanoscale and stochastic analog memory synapses. For the first time, the potential of analog memory synapses to generate precisely timed spikes in SNNs is experimentally demonstrated. The experiment targets applications which directly integrates spike encoded signals generated from bio-mimetic sensors with in-memory computing based learning systems to generate precisely timed control signal spikes for neuromorphic actuators. More than 170,000 phase-change memory (PCM) based synapses from our prototype chip were trained based on an event-driven learning rule, to generate spike patterns with more than 85% of the spikes within a 25ms tolerance interval in a 1250ms long spike pattern. We observe that the accuracy is mainly limited by the imprecision related to device programming and temporal drift of conductance values. We show that an array level scaling scheme can significantly improve the retention of the trained SNN states in the presence of conductance drift in the PCM. Combining the computational potential of supervised SNNs with the parallel compute power of in-memory computing, this work paves the way for next-generation of efficient brain-inspired systems.
引用
收藏
页数:11
相关论文
共 64 条
[1]   Reducing the Impact of Phase-Change Memory Conductance Drift on the Inference of large-scale Hardware Neural Networks [J].
Ambrogio, S. ;
Gallot, M. ;
Spoon, K. ;
Tsai, H. ;
Mackin, C. ;
Wesson, M. ;
Kariyappa, S. ;
Narayanan, P. ;
Liu, C. -C. ;
Kumar, A. ;
Chen, A. ;
Burr, G. W. .
2019 IEEE INTERNATIONAL ELECTRON DEVICES MEETING (IEDM), 2019,
[2]   Equivalent-accuracy accelerated neural-network training using analogue memory [J].
Ambrogio, Stefano ;
Narayanan, Pritish ;
Tsai, Hsinyu ;
Shelby, Robert M. ;
Boybat, Irem ;
di Nolfo, Carmelo ;
Sidler, Severin ;
Giordano, Massimo ;
Bodini, Martina ;
Farinha, Nathan C. P. ;
Killeen, Benjamin ;
Cheng, Christina ;
Jaoudi, Yassine ;
Burr, Geoffrey W. .
NATURE, 2018, 558 (7708) :60-+
[3]   Unsupervised Learning by Spike Timing Dependent Plasticity in Phase Change Memory (PCM) Synapses [J].
Ambrogio, Stefano ;
Ciocchini, Nicola ;
Laudato, Mario ;
Milo, Valerio ;
Pirovano, Agostino ;
Fantini, Paolo ;
Ielmini, Daniele .
FRONTIERS IN NEUROSCIENCE, 2016, 10
[4]  
Anwani N, 2015, IEEE IJCNN
[5]   Temporal precision of spike trains in extrastriate cortex of the behaving macaque monkey [J].
Bair, W ;
Koch, C .
NEURAL COMPUTATION, 1996, 8 (06) :1185-1202
[6]   The structure and precision of retinal spike trains [J].
Berry, MJ ;
Warland, DK ;
Meister, M .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 1997, 94 (10) :5411-5416
[7]   Error-backpropagation in temporally encoded networks of spiking neurons [J].
Bohte, SM ;
Kok, JN ;
La Poutré, H .
NEUROCOMPUTING, 2002, 48 :17-37
[8]   A physics-based model of electrical conduction decrease with time in amorphous Ge2Sb2Te5 [J].
Boniardi, M. ;
Redaelli, A. ;
Pirovano, A. ;
Tortorelli, I. ;
Ielmini, D. ;
Pellizzer, F. .
JOURNAL OF APPLIED PHYSICS, 2009, 105 (08)
[9]   Physical origin of the resistance drift exponent in amorphous phase change materials [J].
Boniardi, Mattia ;
Ielmini, Daniele .
APPLIED PHYSICS LETTERS, 2011, 98 (24)
[10]  
Boybat I, 2018, 2018 NON-VOLATILE MEMORY TECHNOLOGY SYMPOSIUM (NVMTS 2018)