Convergence. of controlled models and finite-state approximation for discounted continuous-time Markov decision processes with constraints

In this paper we consider the convergence of a sequence {M-n} of the models of discounted continuoustime constrained Markov decision processes (MDP) to the "limit" one, denoted by M-infinity. For the models with denumerable states and unbounded transition rates, under reasonably mild conditions we prove that the (constrained) optimal policies and the optimal values of {M-n} converge to those of M-infinity, respectively, using a technique of occupation measures. As an application of the convergence result developed here, we show that an optimal policy and the optimal value for countable-state continuous-time MDP can be approximated by those of finite-state continuous-time MDP. Finally, we further illustrate such finite-state approximation by solving numerically a controlled birth-and-death system and also give the corresponding error bound of the approximation. (C) 2014 Elsevier B.V. All rights reserved.