Further remarks on absorbing Markov decision processes

被引:0
作者
Zhang, Yi [1 ]
Zheng, Xinran [1 ]
机构
[1] Univ Birmingham, Sch Math, Birmingham B15 2TT, England
关键词
Absorbing Markov decision processes; Counterexamples; w-convergence; ws-convergence; POLICIES;
D O I
10.1016/j.orl.2024.107191
中图分类号
C93 [管理学]; O22 [运筹学];
学科分类号
070105 ; 12 ; 1201 ; 1202 ; 120202 ;
摘要
In this note, based on the recent remarkable results of Dufour and Prieto-Rumeau, we deduce that for an absorbing Markov decision process with a given initial state, under a standard compactness-continuity condition, the space of occupation measures has the same convergent sequences, when it is endowed with the weak topology and with the weak-strong topology. We provided two examples demonstrating that imposed condition cannot be replaced with its popular alternative, and the above assertion does not hold for the space of marginals of occupation measures on the state space. Moreover, the examples also clarify some results in the previous literature.
引用
收藏
页数:7
相关论文
共 14 条
[1]  
Altman E., 1999, Constrained Markov Decision Processes, V7, DOI 10.1201/9781315140223
[2]   On ws-convergence of product measures [J].
Balder, EJ .
MATHEMATICS OF OPERATIONS RESEARCH, 2001, 26 (03) :494-518
[3]  
Bertsekas D.P., 1996, Neuro-Dynamic Programming
[4]  
Bogachev V, 2007, Measure Theory, V1
[5]  
Bogachev V. I., 2018, Weak Convergence of Measures
[6]  
Dufour F, 2024, Arxiv, DOI arXiv:2402.10672
[7]   Absorbing Markov decision processes [J].
Dufour, Francois ;
Prieto-Rumeau, Tomas .
ESAIM-CONTROL OPTIMISATION AND CALCULUS OF VARIATIONS, 2024, 30
[8]   SUFFICIENCY OF DETERMINISTIC POLICIES FOR ATOMLESS DISCOUNTED AND UNIFORMLY ABSORBING MDPs WITH MULTIPLE CRITERIA [J].
Feinberg, Eugene A. ;
Piunovskiy, Alexey .
SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2019, 57 (01) :163-191
[9]   Splitting Randomized Stationary Policies in Total-Reward Markov Decision Processes [J].
Feinberg, Eugene A. ;
Rothblum, Uriel G. .
MATHEMATICS OF OPERATIONS RESEARCH, 2012, 37 (01) :129-153
[10]   COMPACTNESS AND SEQUENTIAL COMPACTNESS IN SPACES OF MEASURES [J].
GANSSLER, P .
ZEITSCHRIFT FUR WAHRSCHEINLICHKEITSTHEORIE UND VERWANDTE GEBIETE, 1971, 17 (02) :124-&