A policy improvement method in constrained stochastic dynamic programming

被引：9

作者：

Chang, Hyeong Soo ^{[1
]}

机构：

[1] Sogang Univ, Dept Comp Sci & Engn, Seoul 121742, South Korea

[2] Sogang Univ, Program Integrated Biotechnol, Seoul 121742, South Korea

来源：

IEEE TRANSACTIONS ON AUTOMATIC CONTROL | 2006年 / 51卷 / 09期

关键词：

constrained Markov decision process; dynamic programming; policy improvement; policy iteration;

D O I：

10.1109/TAC.2006.880801

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

This note presents a formal method of improving a given base-policy such that the performance of the resulting policy is no worse than that of the base-policy at all states in constrained stochastic dynamic programming. We consider finite horizon and discounted infinite horizon cases. The improvement method induces a policy iteration-type algorithm that converges to a local optimal policy.

引用

页码：1523 / 1526

页数：4

共 9 条

[1] Altman E, 1998, CONSTRAINED MARKOV D
[2] Bertsekas DP, 2005, LIDS2646 MIT
[3] Parallel rollout for online solution of partially observable Markov decision processes
Chang, HS
Givan, R
Chong, EKP
[J]. DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS, 2004, 14 (03): : 309 - 341
[4] Dynamic programming equations for discounted constrained stochastic control
Chen, RC
Blankenship, GL
[J]. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2004, 49 (05) : 699 - 709
[5] Constrained discounted dynamic programming
Feinberg, EA
Shwartz, A
[J]. MATHEMATICS OF OPERATIONS RESEARCH, 1996, 21 (04) : 922 - 945
[6] Constrained dynamic programming with two discount factors: Applications and an algorithm
Feinberg, EA
Shwartz, A
[J]. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 1999, 44 (03) : 628 - 631
[7] CHARACTERIZATION OF OPTIMAL POLICIES IN VECTOR-VALUED MARKOVIAN DECISION-PROCESSES
FURUKAWA, N
[J]. MATHEMATICS OF OPERATIONS RESEARCH, 1980, 5 (02) : 271 - 279
[8] Puterman M.L., 2008, Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley Series in Probability and Statistics
[9] VECTOR-VALUED MARKOV DECISION-PROCESSES AND THE SYSTEMS OF LINEAR INEQUALITIES
WAKUTA, K
[J]. STOCHASTIC PROCESSES AND THEIR APPLICATIONS, 1995, 56 (01) : 159 - 169

← 1 →