SPSA for Layer-Wise Training of Deep Networks

被引：2

作者：

Wulff, Benjamin ^{[1
,2
]}

Schuecker, Jannis ^{[1
,2
]}

Bauckhage, Christian ^{[1
,2
,3
]}

机构：

[1] Fraunhofer Ctr Machine Learning, St Augustin, Germany

[2] Fraunhofer IAIS, St Augustin, Germany

[3] Univ Bonn, B IT, Bonn, Germany

来源：

ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2018, PT III | 2018年 / 11141卷

关键词：

D O I：

10.1007/978-3-030-01424-7_55

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Concerned with neural learning without backpropagation, we investigate variants of the simultaneous perturbation stochastic approximation (SPSA) algorithm. Experimental results suggest that these allow for the successful training of deep feed-forward neural networks using forward passes only. In particular, we find that SPSA-based algorithms which update network parameters in a layer-wise manner are superior to variants which update all weights simultaneously.

引用

页码：564 / 573

页数：10

共 50 条

[1] Voice Conversion Using Deep Neural Networks With Layer-Wise Generative Training
Chen, Ling-Hui
Ling, Zhen-Hua
Liu, Li-Juan
Dai, Li-Rong
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2014, 22 (12) : 1859 - 1872
[2] Supervised Greedy Layer-Wise Training for Deep Convolutional Networks with Small Datasets
Rueda-Plata, Diego
Ramos-Pollan, Raul
Gonzalez, Fabio A.
COMPUTATIONAL COLLECTIVE INTELLIGENCE (ICCCI 2015), PT I, 2015, 9329 : 275 - 284
[3] Layer-Wise Compressive Training for Convolutional Neural Networks
Grimaldi, Matteo
Tenace, Valerio
Calimera, Andrea
FUTURE INTERNET, 2019, 11 (01)
[4] Layer-Wise Weight Decay for Deep Neural Networks
Ishii, Masato
Sato, Atsushi
IMAGE AND VIDEO TECHNOLOGY (PSIVT 2017), 2018, 10749 : 276 - 289
[5] Stochastic Layer-Wise Precision in Deep Neural Networks
Lacey, Griffin
Taylor, Graham W.
Areibi, Shawki
UNCERTAINTY IN ARTIFICIAL INTELLIGENCE, 2018, : 663 - 672
[6] Unsupervised Layer-Wise Model Selection in Deep Neural Networks
Ludovic, Arnold
Helene, Paugam-Moisy
Michele, Sebag
ECAI 2010 - 19TH EUROPEAN CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2010, 215 : 915 - 920
[7] Layer-Wise Training to Create Efficient Convolutional Neural Networks
Zeng, Linghua
Tian, Xinmei
NEURAL INFORMATION PROCESSING (ICONIP 2017), PT II, 2017, 10635 : 631 - 641
[8] Collaborative Layer-Wise Discriminative Learning in Deep Neural Networks
Jin, Xiaojie
Chen, Yunpeng
Dong, Jian
Feng, Jiashi
Yan, Shuicheng
COMPUTER VISION - ECCV 2016, PT VII, 2016, 9911 : 733 - 749
[9] Multithreaded Layer-wise Training of Sparse Deep Neural Networks using Compressed Sparse Column
Mofrad, Mohammad Hasanzadeh
Melhem, Rami
Ahmad, Yousuf
Hammoud, Mohammad
2019 IEEE HIGH PERFORMANCE EXTREME COMPUTING CONFERENCE (HPEC), 2019,
[10] GREEDY LAYER-WISE TRAINING OF LONG SHORT TERM MEMORY NETWORKS
Xu, Kaisheng
Shen, Xu
Yao, Ting
Tian, Xinmei
Mei, Tao
2018 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA & EXPO WORKSHOPS (ICMEW 2018), 2018,

← 1 2 3 4 5 →