Stochastic Implementation of the Activation Function for Artificial Neural Networks

被引：0
|
作者：
Yeo, Injune [1 ]
Gi, Sang-gyun [1 ]
Lee, Byung-geun [1 ]
Chu, Myonglae [2 ]
机构：
[1] Gwangju Inst Sci & Technol, Sch Elect Engn & Comp Sci, Gwangju, South Korea
[2] IMEC, Interuniv Microelect Ctr, Imager SoC Team, Leuven, Belgium
来源：
PROCEEDINGS OF 2016 IEEE BIOMEDICAL CIRCUITS AND SYSTEMS CONFERENCE (BIOCAS) | 2016年
基金：
新加坡国家研究基金会;
关键词：
Artificial neural network; nonlinear activation function; neuromorphic; stochastic neuron; ananlog computing element;
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
One of the key elements in an artificial neural networks (ANNs) is the activation function (AF), that converts the weighted sum of a neuron's input into a probability of firing rate. The hardware implementation of the AF requires complicated circuits and involves a considerable amount of power dissipation. This renders the integration of a number of neurons onto a single chip difficult. This paper presents circuit techniques for realizing four different types of AFs, such as the step, identity, rectified-linear unit (ReLU), and the sigmoid, based on stochastic computing. The proposed AF circuits are simpler and consume considerably lesser power than the existing ones. A handwritten digit recognition system employing the AF circuits has been simulated for verifying the effectiveness of the techniques.
引用
收藏
页码：440 / 443
页数：4
相关论文
共 50 条
  • [1] A novel type of activation function in artificial neural networks: Trained activation function
    Ertugrul, Omer Faruk
    NEURAL NETWORKS, 2018, 99 : 148 - 157
  • [2] Efficient VLSI Implementation of Neural Networks With Hyperbolic Tangent Activation Function
    Zamanlooy, Babak
    Mirhassani, Mitra
    IEEE TRANSACTIONS ON VERY LARGE SCALE INTEGRATION (VLSI) SYSTEMS, 2014, 22 (01) : 39 - 48
  • [3] A STOCHASTIC TRAINING ALGORITHM FOR ARTIFICIAL NEURAL NETWORKS
    BARTLETT, EB
    NEUROCOMPUTING, 1994, 6 (01) : 31 - 43
  • [4] Accuracy Analysis of Node Activation Function Based on Hardware Implementation of Artificial Neural Network
    Jiang, Nan
    Hou, Ligang
    Guo, Jia
    Zhang, Xinyi
    Lv, Ang
    2018 3RD IEEE INTERNATIONAL CONFERENCE ON INTEGRATED CIRCUITS AND MICROSYSTEMS (ICICM), 2018, : 278 - 281
  • [5] Development of stochastic artificial neural networks for hydrological prediction
    Kingston, GB
    Lambert, MF
    Maier, HR
    MODSIM 2003: INTERNATIONAL CONGRESS ON MODELLING AND SIMULATION, VOLS 1-4: VOL 1: NATURAL SYSTEMS, PT 1; VOL 2: NATURAL SYSTEMS, PT 2; VOL 3: SOCIO-ECONOMIC SYSTEMS; VOL 4: GENERAL SYSTEMS, 2003, : 837 - 842
  • [6] Voltage-to-Voltage Sigmoid Neuron Activation Function Design for Artificial Neural Networks
    Moposita, Tatiana
    Trojman, Lionel
    Crupi, Felice
    Lanuzza, Marco
    Vladimirescu, Andrei
    2022 IEEE 13TH LATIN AMERICAN SYMPOSIUM ON CIRCUITS AND SYSTEMS (LASCAS), 2022, : 164 - 167
  • [7] Function approximation using artificial neural networks
    Zainuddin, Zarita
    Pauline, Ong
    APPLIED MATHEMATICS FOR SCIENCE AND ENGINEERING, 2007, : 140 - +
  • [8] An Efficient Hardware Implementation of Artificial Neural Network based on Stochastic Computing
    Duy-Anh Nguyen
    Huy-Hung Ho
    Duy-Hieu Bui
    Xuan-Tu Tran
    PROCEEDINGS OF 2018 5TH NAFOSTED CONFERENCE ON INFORMATION AND COMPUTER SCIENCE (NICS 2018), 2018, : 237 - 242
  • [9] The Implementation of Artificial Neural Networks Applying to Software Reliability Modeling
    Lo, Jung-Hua
    CCDC 2009: 21ST CHINESE CONTROL AND DECISION CONFERENCE, VOLS 1-6, PROCEEDINGS, 2009, : 4349 - 4354
  • [10] Implementation of Artificial Neural Networks Using Magnetoresistive Random-Access Memory-Based Stochastic Computing Units
    Shao, Yixin
    Sinaga, Sisilia Lamsari
    Sunmola, Idris O.
    Borland, Andrew S.
    Carey, Matthew J.
    Katine, Jordan A.
    Lopez-Dominguez, Victor
    Amiri, Pedram Khalili
    IEEE MAGNETICS LETTERS, 2021, 12
12345