Low complexity probability-based piecewise linear approximation of the sigmoid function

被引：0
|
作者：
Nguyen V.-T. [1 ]
Cai J. [1 ]
Wei L. [1 ]
Chu J. [1 ]
机构：
[1] School of Microelectronics, Xidian University, Xi'an
来源：
| 1600年 / Science Press卷 / 47期
关键词：
Field programmable gate array; Neural networks; Piecewise linear approximation; Probability; Sigmoid function;
D O I：
10.19665/j.issn1001-2400.2020.03.008
中图分类号：
学科分类号：
摘要：
In order to improve the network recognition accuracy in the low complexity condition, a piecewise linear sigmoid function approximation based on the distribution probability of the neurons' values is proposed only with one addition circuit. The sigmoid function is first divided into three fixed regions. Second, according to the neurons' values distribution probability, the curve in each region is segmented into sub-regions to reduce the approximation error and improve the recognition accuracy. The slope of the piecewise linear function is set as 2-n, effectively reducing the hardware implementation complexity. Experiments performed on Xilinx's FPGA-XC7A200T implement the MNIST handwritten digits recognition. The results show that the proposed method achieves a 97.45% recognition accuracy in a deep neural network and 98.42% in a convolutional neural network, up to 0.84% and 0.57% higher than other approximation methods only with one addition circuit. © 2020, The Editorial Board of Journal of Xidian University. All right reserved.
引用
收藏
页码：58 / 65
页数：7
相关论文
共 16 条
  • [1] KANG M, LIM S, GONUGONDLA S, Et al., An In-memory VLSI Architecture for Convolutional Neural Networks, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 8, 3, pp. 494-505, (2018)
  • [2] LIU Q, LIU J, SANG R., Et al., Fast Neural Network Training on FPGA Using Quasi-Newton Optimization Method, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 26, 8, pp. 1575-1579, (2018)
  • [3] HAJDUK Z., Hardware Implementation of Hyperbolic Tangent and Sigmoid Activation Function, Bulletin of the Polish Academy of Sciences: Technical Sciences, 66, 5, pp. 563-577, (2018)
  • [4] DEL CAMPO I, FINKER R, ECHANOBE J, Et al., Controlled Accuracy Approximation of Sigmoid Function for Efficient FPGA-based Implementation of Artificial Neurons, Electronics Letters, 49, 25, pp. 1598-1600, (2013)
  • [5] NAMIN A H, LEBOEUF K, MUSCEDERE R, Et al., Efficient Hardware Implementation of the Hyperbolic Tangent Sigmoid Function, Proceedings of the IEEE International Symposium on Circuits and Systems, pp. 2117-2120, (2009)
  • [6] TIWARI V, KHARE N., Hardware Implementation of Neural Network with Sigmoidal Activation Functions Using CORDIC, Microprocessors and Microsystems, 39, 6, pp. 373-381, (2015)
  • [7] NASCIMENTO I, JARDIM R, MORGADO D F., A New Solution to the Hyperbolic Tangent Implementation in Hardware: Polynomial Modeling of the Fractional Exponential Part, Neural Computing and Applications, 23, 2, pp. 363-369, (2013)
  • [8] ARMATO A, FANUCCI L, SCILINGO E P, Et al., Low-error Digital Hardware Implementation of Artificial Neuron Activation Functions and Their Derivative, Microprocessors and Microsystems, 35, 6, pp. 557-567, (2011)
  • [9] BAJGER M, OMONDI A., Low-error, High-speed Approximation of the Sigmoid Function for Large FPGA Implementations, Journal of Signal Processing Systems, 52, 2, pp. 137-151, (2008)
  • [10] SAVICH A W, MOUSSA M, AREIBI S., The Impact of Arithmetic Representation on Implementing MLP-BP on FPGAs: a Study, IEEE Transactions on Neural Networks, 18, 1, pp. 240-252, (2007)
12