Adaptive stochastic approximation algorithm

被引：4

作者：

Kresoja, Milena ^{[1
]}

Luzanin, Zorana ^{[1
]}

Stojkovska, Irena ^{[2
]}

机构：

[1] Univ Novi Sad, Fac Sci, Dept Math & Informat, Trg Dositeja Obradovica 4, Novi Sad 21000, Serbia

[2] Ss Cyril & Methodius Univ, Fac Nat Sci & Math, Dept Math, Arhimedova 3, Skopje 1000, Macedonia

来源：

NUMERICAL ALGORITHMS | 2017年 / 76卷 / 04期

关键词：

Unconstrained optimization; Stochastic optimization; Stochastic approximation; Noisy function; Adaptive step size; Gradient method; Descent direction; OPTIMIZATION METHODS; GRADIENT;

D O I：

10.1007/s11075-017-0290-4

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure convergence is established. The case when negative gradient is chosen as a search direction is also considered. The algorithm is tested on a set of standard test problems. Numerical results show good performance and verify efficiency of the algorithm compared to some of existing algorithms with adaptive step sizes.

引用

页码：917 / 937

页数：21