Stability of Markov modulated discrete-time dynamic systems

Motivated by various control applications, this paper develops stability analysis of discrete-time systems with regime switching, in which the dynamics are modulated by Markov chains with two time scales. Using the high contrast of the different transition rates among the Markovian states, singularly perturbed Markov chains are used in the formulations. Taking into consideration of the regime changes and time-scale separation makes the stability analysis a difficult task. In this work, asymptotic stability analysis is carried out using perturbed Liapunov function techniques. It is demonstrated that if the limit system is stable, then the original system is also stable. In addition, we examine path excursion, derive bounds on mean recurrence time, and obtain the associated probability bounds. (C) 2003 Elsevier Ltd. All rights reserved.