On the total reward variance for continuous-time Markov reward chains

被引：15
作者：
Van Dijk, Nico M.
Sladky, Karel
机构：
[1] Univ Amsterdam, Dept Econ Sci & Econometr, NL-1018 WB Amsterdam, Netherlands
[2] Acad Sci Czech Republ, Inst Informat Theory & Automat, CR-18208 Prague 8, Czech Republic
来源：
JOURNAL OF APPLIED PROBABILITY | 2006年 / 43卷 / 04期
关键词：
continuous-time Markov reward chain; variance of cumulative reward; asymptotic behaviour; uniformization;
D O I：
10.1239/jap/1165505206
中图分类号：
O21 [概率论与数理统计]; C8 [统计学];
学科分类号：
020208 ; 070103 ; 0714 ;
摘要：
As an extension of the discrete-time case, this note investigates the valiance of the total cumulative reward for continuous-time Markov reward chains with finite state spaces. The results correspond to discrete-time results. In particular, the variance growth rate is shown to be asymptotically linear in time. Expressions are provided to compute this growth rate.
引用
收藏
页码：1044 / 1052
页数：9
相关论文
共 20 条
[1]  
Benito F., 1982, TRABAJOS ESTADISTICA, V33, P73, DOI [10.1007/BF02888435, DOI 10.1007/BF02888435]
[2]  
Dynkin E.B., 1965, MARKOV PROCESSES, V1
[3]   VARIANCE-PENALIZED MARKOV DECISION-PROCESSES [J].
FILAR, JA ;
KALLENBERG, LCM ;
LEE, HM .
MATHEMATICS OF OPERATIONS RESEARCH, 1989, 14 (01) :147-161
[4]   ON FINDING OPTIMAL POLICIES FOR MARKOV DECISION CHAINS - A UNIFYING FRAMEWORK FOR MEAN-VARIANCE-TRADEOFFS [J].
HUANG, Y ;
KALLENBERG, LCM .
MATHEMATICS OF OPERATIONS RESEARCH, 1994, 19 (02) :434-448
[5]   MARKOV DECISION PROCESSES WITH A NEW OPTIMALITY CRITERION - CONTINUOUS TIME [J].
JAQUETTE, SC .
ANNALS OF STATISTICS, 1975, 3 (02) :547-553
[6]   UTILITY CRITERION FOR MARKOV DECISION-PROCESSES [J].
JAQUETTE, SC .
MANAGEMENT SCIENCE, 1976, 23 (01) :43-49
[7]   MARKOV DECISION PROCESSES WITH A NEW OPTIMALITY CRITERION - DISCRETE-TIME [J].
JAQUETTE, SC .
ANNALS OF STATISTICS, 1973, 1 (03) :496-505
[8]   MARKOV DECISION PROCESSES WITH A NEW OPTIMALITY CRITERION - SMALL INTEREST RATES [J].
JAQUETTE, SC .
ANNALS OF MATHEMATICAL STATISTICS, 1972, 43 (06) :1894-1901
[9]  
Kadota Y., 1997, Bulletin of Informatics and Cybernetics, V29, P83
[10]   A VARIANCE MINIMIZATION PROBLEM FOR A MARKOV DECISION-PROCESS [J].
KAWAI, H .
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH, 1987, 31 (01) :140-145
12