Probabilistic design of optimal sequential decision-making algorithms in learning and control

被引:12
作者
Garrabe, Emiland [1 ]
Russo, Giovanni [1 ]
机构
[1] Univ Salerno, Dept Informat & Elect Engn & Appl Math, Salerno, Italy
基金
英国科研创新办公室;
关键词
Sequential decision-making; Data-driven control; Learning; Densities optimization; PATH-INTEGRAL CONTROL; KULLBACK-LEIBLER APPROXIMATION; PREDICTIVE CONTROL; REINFORCEMENT; MODEL; INFORMATION; DENSITY; SYSTEMS; OPTIMIZATION; PERFORMANCE;
D O I
10.1016/j.arcontrol.2022.09.003
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This survey is focused on certain sequential decision-making problems that involve optimizing over probability functions. We discuss the relevance of these problems for learning and control. The survey is organized around a framework that combines a problem formulation and a set of resolution methods. The formulation consists of an infinite-dimensional optimization problem. The methods come from approaches to search optimal solutions in the space of probability functions. Through the lenses of this overarching framework we revisit popular learning and control algorithms, showing that these naturally arise from suitable variations on the formulation mixed with different resolution methods. A running example, for which we make the code available, complements the survey. Finally, a number of challenges arising from the survey are also outlined.
引用
收藏
页码:81 / 102
页数:22
相关论文
共 223 条
[1]  
Agarwal A, 2014, PR MACH LEARN RES, V32, P1638
[2]  
Agrawal S., 2013, INT C MACHINE LEARNI, V28, P127, DOI DOI 10.5555/3042817.3043073
[3]   Offline Contextual Multi-armed Bandits for Mobile Health Interventions: A Case Study on Emotion Regulation [J].
Ameko, Mawulolo K. ;
Beltzer, Miranda L. ;
Cai, Lihua ;
Boukhechba, Mehdi ;
Teachman, Bethany A. ;
Barnes, Laura E. .
RECSYS 2020: 14TH ACM CONFERENCE ON RECOMMENDER SYSTEMS, 2020, :249-258
[4]  
Amos Brandon, 2021, Learning for Dynamics and Control, P6
[5]  
Anastassacos N, 2020, AAAI CONF ARTIF INTE, V34, P7047
[6]   A Fokker-Planck control framework for multidimensional stochastic processes [J].
Annunziato, M. ;
Borzi, A. .
JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2013, 237 (01) :487-507
[7]   Optimal Control of Probability Density Functions of Stochastic Processes [J].
Annunziato, M. ;
Borzi, A. .
MATHEMATICAL MODELLING AND ANALYSIS, 2010, 15 (04) :393-407
[8]  
Annunziato M., 2012, NONLINEAR DYNAMICS E, P1
[9]   A Fokker-Planck control framework for stochastic systems [J].
Annunziato, Mario ;
Borzi, Alfio .
EMS SURVEYS IN MATHEMATICAL SCIENCES, 2018, 5 (1-2) :65-98
[10]  
[Anonymous], 2017, P 31 INT C NEUR INF