On mean reward variance in semi-Markov processes

被引：0

作者：

Karel Sladký

机构：

[1] Academy of Sciences of the Czech Republic,Institute of Information Theory and Automation

来源：

Mathematical Methods of Operations Research | 2005年 / 62卷

关键词：

Markov and semi-Markov processes with rewards; Variance of cumulative reward; Asymptotic behaviour; Primary 90C47; Secondary 60J27;

D O I：

暂无

中图分类号：

学科分类号：

摘要：

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for the embedded Markov chain of semi-Markov processes. Under the assumption that the chain is aperiodic and contains a single class of recurrent states recursive formulae for the variance are obtained which show that the variance growth rate is asymptotically linear in time. Expressions are provided to compute this growth rate.

引用

页码：387 / 397

页数：10

共 16 条

[1] Benito F(1982)Calculating the variance in Markov processes with random reward Trabajos Estadistica Investigacion Operativa 33 73-85
[2] Filar J(1989)Variance penalized Markov decision processes Math Oper Res 14 147-161
[3] Kallenberg LCM(1994)On finding optimal policies for Markov decision chains: a unifying framework for mean-variance-tradeoffs Math Oper Res 19 434-448
[4] Lee H-M(1972)Markov decision processes with a new optimality criterion: small interest rates Ann Math Stat 43 1894-1901
[5] Huang Y(1973)Markov decision processes with a new optimality criterion: discrete time Ann Statist 1 496-505
[6] Kallenberg LCM(1975)Markov decision processes with a new optimality criterion: continuous time Ann Stat 3 547-553
[7] Jaquette SC(1997)A minimum average-variance in Markov decision processes Bull Inform Cybern 29 83-89
[8] Jaquette SC(1987)A variance minimization problem for a Markov decision process Eur J Oper Res 31 140-145
[9] Jaquette SC(1987)Markov decision processes with a minimum-variance criterion J Math Anal Appl 123 572-583
[10] Kadota Y(1971)On the variance in controlled Markov chains Kybernetika 7 1-12

← 1 2 →