All-optical ultrafast ReLU function for energy-efficient nanophotonic deep learning

被引:48
作者
Li, Gordon H. Y. [1 ]
Sekine, Ryoto [2 ]
Nehra, Rajveer [2 ]
Gray, Robert M. [2 ]
Ledezma, Luis [1 ,3 ]
Guo, Qiushi [2 ]
Marandi, Alireza [1 ,2 ]
机构
[1] CALTECH, Dept Appl Phys, Pasadena, CA 91125 USA
[2] CALTECH, Dept Elect Engn, Pasadena, CA 91125 USA
[3] CALTECH, Jet Prop Lab, Pasadena, CA 91125 USA
基金
美国国家科学基金会;
关键词
deep learning; optical computing; optical neural networks; thin-film lithium niobate; ACTIVATION FUNCTION; NEURAL-NETWORKS; PERFORMANCE; CONVERSION;
D O I
10.1515/nanoph-2022-0137
中图分类号
TB3 [工程材料学];
学科分类号
0805 ; 080502 ;
摘要
In recent years, the computational demands of deep learning applications have necessitated the introduction of energy-efficient hardware accelerators. Optical neural networks are a promising option; however, thus far they have been largely limited by the lack of energy-efficient nonlinear optical functions. Here, we experimentally demonstrate an all-optical Rectified Linear Unit (ReLU), which is the most widely used nonlinear activation function for deep learning, using a periodically-poled thin-film lithium niobate nanophotonic waveguide and achieve ultra-low energies in the regime of femtojoules per activation with near-instantaneous operation. Our results provide a clear and practical path towards truly all-optical, energy-efficient nanophotonic deep learning.
引用
收藏
页码:847 / 855
页数:9
相关论文
共 52 条
[1]   ITO-based electro-absorption modulator for photonic neural activation function [J].
Amin, R. ;
George, J. K. ;
Sun, S. ;
de Lima, T. Ferreira ;
Tait, A. N. ;
Khurgin, J. B. ;
Miscuglio, M. ;
Shastri, B. J. ;
Prucnal, P. R. ;
El-Ghazawi, T. ;
Sorger, V. J. .
APL MATERIALS, 2019, 7 (08)
[2]  
Ashtiani F., 2021, ARXIV PREPRINT ARXIV
[3]   Hardware error correction for programmable photonics [J].
Bandyopadhyay, Saumil ;
Hamerly, Ryan ;
Englund, Dirk .
OPTICA, 2021, 8 (10) :1247-1255
[4]  
Brown TB, 2020, ADV NEUR IN, V33
[5]   NVIDIA A100 Tensor Core GPU: Performance and Innovation [J].
Choquette, Jack ;
Gandhi, Wishwesh ;
Giroux, Olivier ;
Stam, Nick ;
Krashinsky, Ronny .
IEEE MICRO, 2021, 41 (02) :29-35
[6]  
Clevert D.-A., 2015, Fast and Accurate Deep Network Learning by Exponential Linear Units ELUs
[7]   Optical and electrical programmable computing energy use comparison [J].
Cole, Chris .
OPTICS EXPRESS, 2021, 29 (09) :13153-13170
[8]  
Crnjanski J., 2006, OPT LETT, V46, P20032021
[9]   All-optical reservoir computer based on saturation of absorption [J].
Dejonckheere, Antoine ;
Duport, Francois ;
Smerieri, Anteo ;
Fang, Li ;
Oudar, Jean-Louis ;
Haelterman, Marc ;
Massar, Serge .
OPTICS EXPRESS, 2014, 22 (09) :10868-10881
[10]   Fully analogue photonic reservoir computer [J].
Duport, Francois ;
Smerieri, Anteo ;
Akrout, Akram ;
Haelterman, Marc ;
Massar, Serge .
SCIENTIFIC REPORTS, 2016, 6