First Passage Risk Probability Minimization for Piecewise Deterministic Markov Decision Processes

被引：0
作者：
Xin Wen
Hai-feng Huo
Xian-ping Guo
机构：
[1] Sun Yat-sen University,School of Mathematics
[2] Sun Yat-sen University,Guangdong Province Key Laboratory of Computational Science
[3] Guangxi University of Science and Technology,School of Science
来源：
Acta Mathematicae Applicatae Sinica, English Series | 2022年 / 38卷
关键词：
piecewise deterministic Markov decision processes; risk probability; first passage time; -optimal policy; 90C40; 60J27;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
This paper is an attempt to study the minimization problem of the risk probability of piecewise deterministic Markov decision processes (PDMDPs) with unbounded transition rates and Borel spaces. Different from the expected discounted and average criteria in the existing literature, we consider the risk probability that the total rewards produced by a system do not exceed a prescribed goal during a first passage time to some target set, and aim to find a policy that minimizes the risk probability over the class of all history-dependent policies. Under suitable conditions, we derive the optimality equation (OE) for the probability criterion, prove that the value function of the minimization problem is the unique solution to the OE, and establish the existence of ε(≥ 0)-optimal policies. Finally, we provide two examples to illustrate our results.
引用
收藏
页码：549 / 567
页数：18
相关论文
共 38 条
[1]  
Costa OLV(2016)Constrained and unconstrained optimal discounted control of piecewise deterministic Markov processes SIAM J. Control Optim. 54 1444-1474
[2]  
Dufour F(2020)Low-lying eigenvalues and convergence to the equilibrium of some piecewise deterministic Markov processes generators in the small temperature regime Ann. Henri Poincaré 21 3575-3608
[3]  
Piunovskiy AB(2020)On risk-sensitive piecewise deterministic Markov decision processes Appl. Math. Optim. 81 685-710
[4]  
Guillin A(2011)Discounted continuous-time constrained Markov decision processes in Polish spaces Ann. Appl. Probab. 21 2016-2049
[5]  
Nectoux B(2019)Risk-sensitive continuous-time Markov decision processes with unbounded rates and Borel spaces Discrete Event Dyn. Syst. 294 445-471
[6]  
Guo X(2017)Constrained total undiscounted continuous-time Markov decision processes Bernoulli 23 1694-1736
[7]  
Zhang Y(2015)A minimization problem of the risk probability in first passage semi-Markov decision processes with loss rates Sci. China Math. 58 1923-1938
[8]  
Guo XP(2019)Finite-horizon piecewise deterministic Markov decision processes with unbounded transition rates Stochastics 91 67-95
[9]  
Song XY(2020)Risk-sensitive finite-horizon piecewise deterministic Markov decision processes Oper. Res. Lett. 48 96-103
[10]  
Guo XP(2020)Risk probability minimization problems for continuous-time Markov decision processes on finite horizon IEEE Trans. Automat. Control 65 3199-3206
1234