A Cyclical Learning Rate Method in Deep Learning Training

被引：11

作者：

Li, Jiaqi ^{[1
]}

Yang, Xiaodong ^{[1
]}

机构：

[1] ZhejiangGongshang Univ, Sch Informat & Elect Engn, Hangzhou, Peoples R China

来源：

PROCEEDINGS OF THE 2020 INTERNATIONAL CONFERENCE ON COMPUTER, INFORMATION AND TELECOMMUNICATION SYSTEMS (CITS) | 2020年

关键词：

Deep learning; Learning rate; Accuracy; Deep neural networks;

D O I：

10.1109/cits49457.2020.9232482

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

The learning rate is an important hyperparameter for training deep neural networks. The traditional learning rate method has the problems of instability of accuracy. Aiming at these problems, we proposed a new learning rate method with different cyclical changes in each training cycle instead of a fixed value. It achieves higher accuracy in less iterations and faster convergence. Through the experiment on CIFAR-10 and CIFAR-100 datasets based on VGG network and RESNET network, the final results show that the proposed method has better results on stability and accuracy than cyclical learning rate method.

引用

页码：140 / 144

页数：5