Stochastic optimal control problems of discrete-time Markov jump systems

In this paper, we consider the indefinite stochastic optimal control problems of discrete-time Markov jump linear systems. Firstly, we establish the new stochastic maximum principle, and by solving the forward-backward stochastic difference equations with Markov jump (FBSDEs-MJ), we derive the necessary and sufficient solvability condition of the indefinite control problem with non-discounted cost, which is in an explicit analytical expression. Then, the optimal control is designed by a series of coupled generalized Riccati difference equations with Markov jump (GRDEs-MJ) and linear recursive equations with Markov jump (LREs-MJ). Moreover, based on the non-discounted cost case, we deduce the optimal control problem with discounted cost. Finally, a numerical example for defined-benefit (DB) pension fund with regime switching is exploited to illustrate the validity of the obtained results.