ACCELERATION OF STOCHASTIC-APPROXIMATION BY AVERAGING

被引：1033
作者：
POLYAK, BT [1 ]
JUDITSKY, AB [1 ]
机构：
[1] INST RECH INFORMAT & SYST ALEATOIRES,F-35042 RENNES,FRANCE
来源：
SIAM JOURNAL ON CONTROL AND OPTIMIZATION | 1992年 / 30卷 / 04期
关键词：
STOCHASTIC APPROXIMATION; RECURSIVE ESTIMATION; STOCHASTIC OPTIMIZATION; OPTIMAL ALGORITHMS;
D O I：
10.1137/0330046
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
A new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated. Convergence with probability one is proved for a variety of classical optimization and identification problems. It is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence.
引用
收藏
页码：838 / 855
页数：18
相关论文
共 50 条
  • [41] Asymptotic behavior of truncated stochastic approximation procedures
    Sharia T.
    Zhong L.
    [J]. Mathematical Methods of Statistics, 2017, 26 (1) : 37 - 54
  • [42] Stochastic approximation algorithms: Overview and recent trends
    B Bharath
    V S Borkar
    [J]. Sadhana, 1999, 24 : 425 - 452
  • [43] Random Directions Stochastic Approximation With Deterministic Perturbations
    Prashanth, L. A.
    Bhatnagar, Shalabh
    Bhavsar, Nirav
    Fu, Michael
    Marcus, Steven, I
    [J]. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2020, 65 (06) : 2450 - 2465
  • [44] A Retrospective Approximation Approach for Smooth Stochastic Optimization
    Newton, David
    Bollapragada, Raghu
    Pasupathy, Raghu
    Yip, Nung Kwan
    [J]. MATHEMATICS OF OPERATIONS RESEARCH, 2024,
  • [45] Stochastic approximation: a survey
    Kushner, Harold
    [J]. WILEY INTERDISCIPLINARY REVIEWS-COMPUTATIONAL STATISTICS, 2010, 2 (01): : 87 - 96
  • [46] Combining the stochastic counterpart and stochastic approximation methods
    Dussault, JP
    Labrecque, D
    LEcuyer, P
    Rubinstein, RY
    [J]. DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS, 1997, 7 (01): : 5 - 28
  • [47] ROBUST STOCHASTIC APPROXIMATION APPROACH TO STOCHASTIC PROGRAMMING
    Nemirovski, A.
    Juditsky, A.
    Lan, G.
    Shapiro, A.
    [J]. SIAM JOURNAL ON OPTIMIZATION, 2009, 19 (04) : 1574 - 1609
  • [48] New stochastic approximation algorithms with adaptive step sizes
    Xu, Zi
    Dai, Yu-Hong
    [J]. OPTIMIZATION LETTERS, 2012, 6 (08) : 1831 - 1846
  • [49] Global random optimization by simultaneous perturbation stochastic approximation
    Maryak, JL
    Chin, DC
    [J]. PROCEEDINGS OF THE 2001 AMERICAN CONTROL CONFERENCE, VOLS 1-6, 2001, : 756 - 762
  • [50] Budget-dependent convergence rate of stochastic approximation
    L'Ecuyer, P
    Yin, G
    [J]. SIAM JOURNAL ON OPTIMIZATION, 1998, 8 (01) : 217 - 247
12345