CHARACTERIZATION OF THE CONDITIONAL STATIONARY DISTRIBUTION IN MARKOV CHAINS VIA SYSTEMS OF LINEAR INEQUALITIES

This paper considers ergodic, continuous-time Markov chains (X(t)}(t is an element of(-infinity, infinity)) on Z(+) = {0, 1, . . . }. For an arbitrarily fixed N is an element of Z(+), we study the conditional stationary distribution pi(N) given the Markov chain being in {0, 1, . . . , N}. We first characterize pi(N) via systems of linear inequalities and identify simplices that contain pi(N), by examining the (N + 1) x (N + 1) northwest corner block of the infinitesimal generator Q and the subset of the first N + 1 states whose members are directly reachable from at least one state in {N + 1, N + 2, . . .}. These results are closely related to the augmented truncation approximation (ATA), and we provide some practical implications for the ATA. Next we consider an extension of the above results, using the (K + 1) x (K + 1) (K > N) northwest corner block of Q and the subset of the first K + 1 states whose members are directly reachable from at least one state in (K + 1, K + 2, . . .}. Furthermore, we introduce new state transition structures called (K, N)-skip-free sets, using which we obtain the minimum convex polytope that contains pi(N).