1S1R Optimization for High-Frequency Inference on Binarized Spiking Neural Networks

被引:12
作者
Lopez, Joel Minguet [1 ]
Rafhay, Quentin [2 ]
Dampfhoffer, Manon [3 ]
Reganaz, Lucas [1 ]
Castellani, Niccolo [1 ]
Meli, Valentina [1 ]
Martin, Simon [1 ]
Grenouillet, Laurent [1 ]
Navarro, Gabriele [1 ]
Magis, Thomas [1 ]
Carabasse, Catherine [1 ]
Hirtzlin, Tifenn [1 ]
Vianello, Elisa [1 ]
Deleruyelle, Damien [4 ]
Portal, Jean-Michel [5 ]
Molas, Gabriel [1 ]
Andrieu, Francois [1 ]
机构
[1] LETI, CEA, MINATEC Campus,17 Rue Martyrs, F-38054 Grenoble, France
[2] Univ Grenoble Alpes, Univ Savoie Mt Blanc, CNRS, Grenoble INO,IMEP LAHC, 3 Parvis Louis Neel, F-38000 Grenoble, France
[3] Univ Grenoble Alpes, CEA, CNRS, Grenoble INP,INAC Spintec, 17 Rue Martyrs, F-38054 Grenoble, France
[4] INSA Lyon, INL CNRS, 7 Ave Jean Capelle, F-69621 Villeurbanne, France
[5] Aix Marseille Univ, CNRS, IM2NP, 5 Rue Enrico Fermi, F-13009 Marseille, France
关键词
crossbar; ovonic threshold switch (OTS); resistive random-access memory (RRAM); spiking neural networks; IN-MEMORY; EFFICIENT; RRAM;
D O I
10.1002/aelm.202200323
中图分类号
TB3 [工程材料学];
学科分类号
0805 ; 080502 ;
摘要
Single memristor crossbar arrays are a very promising approach to reduce the power consumption of deep learning accelerators. In parallel, the emerging bio-inspired spiking neural networks (SNNs) offer very low power consumption with satisfactory performance on complex artificial intelligence tasks. In such neural networks, synaptic weights can be stored in nonvolatile memories. The latter are massively read during inference, which can lead to device failure. In this context, a 1S1R (1 Selector 1 Resistor) device composed of a HfO2-based OxRAM memory stacked on a Ge-Se-Sb-N-based ovonic threshold switch (OTS) back-end selector is proposed for high-density binarized SNNs (BSNNs) synaptic weight hardware implementation. An extensive experimental statistical study combined with a novel Monte Carlo model allows to deeply analyze the OTS switching dynamics based on field-driven stochastic nucleation of conductive dots in the layer. This allows quantifying the occurrence frequency of OTS erratic switching as a function of the applied voltages and 1S1R reading frequency. The associated 1S1R reading error rate is calculated. Focusing on the standard machine learning MNIST image recognition task, BSNN figures of merit (footprint, electrical consumption during inference, frequency of inference, accuracy, and tolerance to errors) are optimized by engineering the network topology, training procedure, and activations sparsity.
引用
收藏
页数:11
相关论文
共 31 条
[1]   Equivalent-accuracy accelerated neural-network training using analogue memory [J].
Ambrogio, Stefano ;
Narayanan, Pritish ;
Tsai, Hsinyu ;
Shelby, Robert M. ;
Boybat, Irem ;
di Nolfo, Carmelo ;
Sidler, Severin ;
Giordano, Massimo ;
Bodini, Martina ;
Farinha, Nathan C. P. ;
Killeen, Benjamin ;
Cheng, Christina ;
Jaoudi, Yassine ;
Burr, Geoffrey W. .
NATURE, 2018, 558 (7708) :60-+
[2]   Neuromorphic computing with multi-memristive synapses [J].
Boybat, Irem ;
Le Gallo, Manuel ;
Nandakumar, S. R. ;
Moraitis, Timoleon ;
Parnell, Thomas ;
Tuma, Tomas ;
Rajendran, Bipin ;
Leblebici, Yusuf ;
Sebastian, Abu ;
Eleftheriou, Evangelos .
NATURE COMMUNICATIONS, 2018, 9
[3]   CMOS-integrated memristive non-volatile computing-in-memory for AI edge processors [J].
Chen, Wei-Hao ;
Dou, Chunmeng ;
Li, Kai-Xiang ;
Lin, Wei-Yu ;
Li, Pin-Yi ;
Huang, Jian-Hao ;
Wang, Jing-Hong ;
Wei, Wei-Chen ;
Xue, Cheng-Xin ;
Chiu, Yen-Cheng ;
King, Ya-Chin ;
Lin, Chorng-Jung ;
Liu, Ren-Shuo ;
Hsieh, Chih-Cheng ;
Tang, Kea-Tiong ;
Yang, J. Joshua ;
Ho, Mon-Shu ;
Chang, Meng-Fan .
NATURE ELECTRONICS, 2019, 2 (09) :420-428
[4]   A 22nm 96KX144 RRAM macro with a self-tracking reference and a low ripple charge pump to achieve a configurable read window and a wide operating voltage range [J].
Chou, Chung-Cheng ;
Lin, Zheng-Jun ;
Lai, Chien-An ;
Su, Chin-, I ;
Tseng, Pei-Ling ;
Chen, Wei-Chi ;
Tsai, Wu-Chin ;
Chu, Wen-Ting ;
Ong, Tong-Chem ;
Chuang, Harry ;
Chih, Yu-Der ;
Chang, Tsung-Yung Jonathan .
2020 IEEE SYMPOSIUM ON VLSI CIRCUITS, 2020,
[5]   IR-QNN Framework: An IR Drop-Aware Offline Training of Quantized Crossbar Arrays [J].
Fouda, Mohammed E. ;
Lee, Sugil ;
Lee, Jongeun ;
Kim, Gun Hwan ;
Kurdahi, Fadi ;
Eltawi, Ahmed M. .
IEEE ACCESS, 2020, 8 :228392-228408
[6]   Outstanding Bit Error Tolerance of Resistive RAM-Based Binarized Neural Networks [J].
Hirtzlin, T. ;
Bocquett, M. ;
Klein, J. O. ;
Nowak, E. ;
Vianello, E. ;
Portal, J. M. ;
Querlioz, D. .
2019 IEEE INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE CIRCUITS AND SYSTEMS (AICAS 2019), 2019, :288-292
[7]   Memristor-Based Analog Computation and Neural Network Classification with a Dot Product Engine [J].
Hu, Miao ;
Graves, Catherine E. ;
Li, Can ;
Li, Yunning ;
Ge, Ning ;
Montgomery, Eric ;
Davila, Noraica ;
Jiang, Hao ;
Williams, R. Stanley ;
Yang, J. Joshua ;
Xia, Qiangfei ;
Strachan, John Paul .
ADVANCED MATERIALS, 2018, 30 (09)
[8]   Efficient and Optimized Methods for Alleviating the Impacts of IR-Drop and Fault in RRAM Based Neural Computing Systems [J].
Huang, Chenglong ;
Xu, Nuo ;
Qiu, Keni ;
Zhu, Yujie ;
Ma, Desheng ;
Fang, Liang .
IEEE JOURNAL OF THE ELECTRON DEVICES SOCIETY, 2021, 9 :645-652
[9]   Emerging neuromorphic devices [J].
Ielmini, Daniele ;
Ambrogio, Stefano .
NANOTECHNOLOGY, 2020, 31 (09)
[10]  
Jain P, 2019, ISSCC DIG TECH PAP I, V62, P212, DOI 10.1109/ISSCC.2019.8662393