THE VANISHING DISCOUNT APPROACH FOR THE AVERAGE CONTINUOUS CONTROL OF PIECEWISE DETERMINISTIC MARKOV PROCESSES

被引:9
作者
Costa, O. L. V. [1 ]
Dufour, F. [2 ,3 ]
机构
[1] Univ Sao Paulo, Dept Engn Telecomunicacoes & Controle, Escola Politecn, BR-05508900 Sao Paulo, Brazil
[2] Univ Bordeaux 1, F-33405 Talence, France
[3] INRIA Bordeaux Sud Ouest, Bordeaux, France
基金
巴西圣保罗研究基金会;
关键词
Piecewise-deterministic Markov process; continuous time; long-run average cost; optimal control; integro-differential optimality inequation; vanishing discount approach; DECISION-PROCESSES; CAPACITY EXPANSION; OPTIMALITY; STABILITY; SPACES;
D O I
10.1239/jap/1261670695
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
This work is concerned with the existence of an optimal control strategy for the long-run average continuous control problem of piecewise-deterministic Markov processes (PDMPs). In Costa and Dufour (2008), sufficient conditions were derived to ensure the existence of an optimal control by using the vanishing discount approach. These conditions were mainly expressed in terms of the relative difference of the alpha-discount value functions. The main goal of this paper is to derive tractable conditions directly related to the primitive data of the PDMP to ensure the existence of an optimal control. The present work can be seen as a continuation of the results derived in Costa and Dufour (2008). Our main assumptions are written in terms of some integro-differential inequalities related to the so-called expected growth condition, and geometric convergence of the post-jump location kernel associated to the PDMP. An example based on the capacity expansion problem is presented, illustrating the possible applications of the results developed in the paper.
引用
收藏
页码:1157 / 1183
页数:27
相关论文
共 16 条
[1]  
[Anonymous], AVERAGE CONTIN UNPUB
[2]   Stability and ergodicity of piecewise deterministic Markov processes [J].
Costa, O. L. V. ;
Dufour, F. .
SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2008, 47 (02) :1053-1077
[3]  
Costa Oswaldo L. V., 2007, Proceedings of the European Control Conference 2007 (ECC), P5052
[4]  
DAVIS MHA, 1984, J ROY STAT SOC B MET, V46, P353
[5]   OPTIMAL CAPACITY EXPANSION UNDER UNCERTAINTY [J].
DAVIS, MHA ;
DEMPSTER, MAH ;
SETHI, SP ;
VERMES, D .
ADVANCES IN APPLIED PROBABILITY, 1987, 19 (01) :156-176
[6]  
Davis MHA., 1993, Markov models and optimization, Monographs on Statistics and Applied Probability
[7]   Stability of piecewise-deterministic Markov processes [J].
Dufour, F ;
Costa, OLV .
SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 1999, 37 (05) :1483-1502
[8]   Average optimality for continuous-time Markov decision processes in Polish spaces [J].
Guo, Xianping ;
Rieder, Ulrich .
ANNALS OF APPLIED PROBABILITY, 2006, 16 (02) :730-756
[9]   Average optimality for Markov decision processes in Borel spaces: A new condition and approach [J].
Guo, Xianping ;
Zhu, Quanxin .
JOURNAL OF APPLIED PROBABILITY, 2006, 43 (02) :318-334
[10]  
Hernandez-Lerma O., 1996, APPL MATH, V30