Stochastic Computing Architectures for Lightweight LSTM Neural Networks

被引：0

作者：

Sengupta, Roshwin ^{[1
]}

Polian, Ilia ^{[1
]}

Hayes, John P. ^{[2
]}

机构：

[1] Univ Stuttgart, Inst Comp Architecture & Comp Engn, Stuttgart, Germany

[2] Univ Michigan, Comp Engn Lab, Ann Arbor, MI 48109 USA

来源：

2022 25TH INTERNATIONAL SYMPOSIUM ON DESIGN AND DIAGNOSTICS OF ELECTRONIC CIRCUITS AND SYSTEMS (DDECS) | 2022年

基金：

美国国家科学基金会;

关键词：

LSTM; recurrent neural nets; stochastic; computing;

D O I：

暂无

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

For emerging edge and near-sensor systems to perform hard classification tasks locally, they must avoid costly communication with the cloud. This requires the use of compact classifiers such as recurrent neural networks of the long short term memory (LSTM) type, as well as a low-area hardware technology such as stochastic computing (SC). We study the benefits and costs of applying SC to LSTM design. We consider a design space spanned by fully binary ( non-stochastic), fully stochastic, and several hybrid (mixed) LSTM architectures, and design and simulate examples of each. Using standard classification benchmarks, we show that area and power can be reduced up to 47% and 86% respectively with little or no impact on classification accuracy. We demonstrate that fully stochastic LSTMs can deliver acceptable accuracy despite accumulated errors. Our results also suggest that ReLU is preferable to tanh as an activation function in stochastic LSTMs

引用

页码：124 / 129

页数：6