Activation Function Dependence of Data-Driven Spectra Prediction of Nanostructures

Accompanied by the fast growth of data and computing power, Deep Learning has developed at a tremendous speed. Other related areas including material science, physics, chemistry, medical science, and engineering also benefit from this well-proven powerful tool. Activation function is an essential part of Deep Neural Networks, which has attracted a lot of attention, but there are few researches based on nanophotonics problems. In this work, comprehensive research is made on the effectiveness of several widely used activation functions. This research shows that among those fancy activation functions that are investigated, Tanhshrink performs the best, which can predict the spectra with a Root Mean Square Error <0.005 for over 99% of these randomly generated instances. Other traditional activation functions like Tanh and Sigmoid also show excellent outcomes in competition with novel ones. This work shows that although deep learning has already been a powerful tool to solve physics problems, there is still a lot of fundamental work to be done to achieve its maximum potential.