Discrete-time switching control in random walks

In this paper we study a discrete-time switching control problem when the dynamic of the system evolves as a random walk. The payoff function is a discounted-type total cost whose discount factor may depend on the state and/or the action variables. This flexibility includes cases when the controller can stop the dynamics of the system whether is optimal to do it. To provide optimality results, we use the well-known dynamic programming method that leads to the study of certain functional equations. Under suitable conditions, we prove the existence of solutions to these equations, and we also show that the optimal value of the control problem becomes a minimal solution of these equations. Finally, we illustrate our results through an example about the control of epidemic processes.