A turnpike theorem for a risk-sensitive Markov decision process with stopping

被引:21
作者
Denardo, Eric V.
Rothblum, Uriel G.
机构
[1] Yale Univ, Ctr Syst Sci, New Haven, CT 06520 USA
[2] Technion Israel Inst Technol, Fac Ind Engn & Management, IL-32000 Haifa, Israel
关键词
Markov control process; risk-sensitive dynamic programming; stopping; turnpike theorem; overtaking optimality;
D O I
10.1137/S0363012904442616
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This paper concerns a Markov decision problem with an infinite planning horizon, stationary data, an exponential utility function, and no discounting, but also with the possibility of voluntary or involuntary termination ( stopping). This paper establishes conditions under which the expected utility of the income received by repeated use of a well-chosen stationary control equals the limit, as n becomes large, of the optimal expected utility for the n-period problem. Tests for these (turnpike) conditions are provided, as is a linear program whose optimal solution determines such a control.
引用
收藏
页码:414 / 431
页数:18
相关论文
共 37 条
[1]  
Bellman R., 1957, DYNAMIC PROGRAMMING
[2]  
BLACKWELL D, 1967, 5TH P BERK S MATH ST, V1, P415
[3]  
BLACKWELL D, 1995, DISCOUNTED DYNAMIC P, V36, P226
[4]   The value iteration algorithm in risk-sensitive average Markov decision chains with finite state space [J].
Cavazos-Cadena, R ;
Montes-de-Oca, R .
MATHEMATICS OF OPERATIONS RESEARCH, 2003, 28 (04) :752-776
[5]  
CAVAZOSCADENA R, IN PRESS J APPL PROB
[6]  
Denardo E. V., 1979, Mathematics of Operations Research, V4, P144, DOI 10.1287/moor.4.2.144
[7]   Totally expanding multiplicative systems [J].
Denardo, EV ;
Rothblum, UG .
LINEAR ALGEBRA AND ITS APPLICATIONS, 2005, 406 :142-158
[8]   CONTRACTION MAPPINGS IN THEORY UNDERLYING DYNAMIC PROGRAMMING [J].
DENARDO, EV .
SIAM REVIEW, 1967, 9 (02) :165-&
[9]   Index policies for stochastic search in a forest with an a application to R&D project management [J].
Denardo, EV ;
Rothblum, UG ;
Van der Heyden, L .
MATHEMATICS OF OPERATIONS RESEARCH, 2004, 29 (01) :162-181
[10]   OPTIMAL STOPPING, EXPONENTIAL UTILITY, AND LINEAR-PROGRAMMING [J].
DENARDO, EV ;
ROTHBLUM, UG .
MATHEMATICAL PROGRAMMING, 1979, 16 (02) :228-244