Drift and monotonicity conditions for continuous-time controlled Markov chains with an average criterion

被引:40
作者
Guo, XP [1 ]
Hernández-Lerma, O
机构
[1] Inst Politecn Nacl, Ctr Invest & Estudios Avanzados, Dept Math, Mexico City 07000, DF, Mexico
[2] Zhongshan Univ, Sch Math & Computat Sci, Guangzhou 510275, Peoples R China
关键词
average (or ergodic) reward/cost criterion; continuous-time controlled Markov chains (or continuous-time Markov decision processes); drift and monotonicity conditions; optimal stationary policy; unbounded transition and reward/cost rates;
D O I
10.1109/TAC.2002.808469
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In this paper, we give conditions for the existence of average optimal policies for continuous-time controlled Markov chains with a denumerable state-space and Borel action sets. The' transition rates are allowed to be unbounded, and the reward/cost rates may have neither upper nor lower bounds. In the spirit of the "drift and monotonicity" conditions-for continuous-time Markov processes, we propose a new set of conditions on the controlled process' primitive data under which the existence of optimal (deterministic) stationary policies in the class of randomized Markov policies is.proved using the extended generator approach instead of Kolmogorov's forward equation used in the previous literature, and.under which the convergence of a policy iteration method is also shown. Moreover, we use a controlled queueing system to show that all of our conditions are satisfied, whereas those in the previous literature fail to hold.
引用
收藏
页码:236 / 245
页数:10
相关论文
共 23 条
[1]  
Anderson W. J., 1991, CONTINUOUS TIME MARK
[2]  
[Anonymous], STOCHASTIC DYNAMIC P
[3]  
[Anonymous], 1960, MARKOV CHAINS STATIO, DOI DOI 10.1007/978-3-642-49686-8
[4]   The relations among potentials, perturbation analysis, and Markov decision processes [J].
Cao, XR .
DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS, 1998, 8 (01) :71-87
[5]  
Davis M., 1993, MARKOV MODELS OPTIMI
[6]  
Dynkin E.B., 1979, Grundlehren der Mathematischen Wissenschaften, V235
[7]   On the integro-differential equations of purely discontinuous Markoff processes [J].
Feller, Willy .
TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY, 1940, 48 (1-3) :488-515
[8]   ON RECURRENT DENUMERABLE DECISION PROCESSES [J].
FISHER, L .
ANNALS OF MATHEMATICAL STATISTICS, 1968, 39 (02) :424-&
[9]   A note on optimality conditions for continuous-time Markov decision processes with average cost criterion [J].
Guo, XP ;
Liu, K .
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2001, 46 (12) :1984-1989
[10]  
GUO XP, 2001, MARKOV PROCESSES CON