A study on a low power optimization algorithm for an edge-AI device

被引：6

作者：

Kaneko, Tatsuya ^{[1
]}

Orimo, Kentaro ^{[1
]}

Hida, Itaru ^{[1
]}

Takamaeda-Yamazaki, Shinya ^{[1
]}

Ikebe, Masayuki ^{[1
]}

Motomura, Masato ^{[1
]}

Asai, Tetsuya ^{[1
]}

机构：

[1] Hokkaido Univ, Grad Sch Informat Sci & Technol IST, Div Elect Informat, Kita Ku, M BLDG 2F,Kita 14,Nishi 9, Sapporo, Hokkaido 0600814, Japan

来源：

IEICE NONLINEAR THEORY AND ITS APPLICATIONS | 2019年 / 10卷 / 04期

关键词：

machine learning; edge AI; training algorithm; backpropagation; quantization; low power; GRADIENT;

D O I：

10.1587/nolta.10.373

中图分类号：

O1 [数学];

学科分类号：

0701 ; 070101 ;

摘要：

Although research on the inference phase of edge artificial intelligence (AI) has made considerable improvement, the required training phase remains an unsolved problem. Neural network (NN) processing has two phases: inference and training. In the training phase, a NN incurs high calculation cost. The number of bits (bitwidth) in the training phase is several orders of magnitude larger than that in the inference phase. Training algorithms, optimized to software, are not appropriate for training hardware-oriented NNs. Therefore, we propose a new training algorithm for edge AI: backpropagation (BP) using a ternarized gradient. This ternarized backpropagation (TBP) provides a balance between calculation cost and performance. Empirical results demonstrate that in a two-class classification task, TBP works well in practice and compares favorably with 16-bit BP (Fixed-BP).

引用

页码：373 / 389

页数：17

共 19 条

[1] [Anonymous], GOOGLE KEYNOTE
[2] [Anonymous], 2016, ABS160606160 ARXIV
[3] [Anonymous], 2002, P 2002 ACM SIGDA 10
[4] [Anonymous], 2017, ICCV
[5] [Anonymous], 2017, Parallel wavenet: Fast high-fidelity speech synthesis
[6] [Anonymous], ADADELTA: An Adaptive Learning Rate Method
[7] [Anonymous], INT SUP C JUN
[8] Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, Leon
[J]. COMPSTAT'2010: 19TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL STATISTICS, 2010, : 177 - 186
[9] Caruana R, 2001, ADV NEUR IN, V13, P402
[10] Dean Jeff., 2017, Machine learning for systems and systems for machine learning

← 1 2 →