Best-Response Dynamics for Evolutionary Stochastic Games

We introduce a version of best-response (BR) dynamics for stochastic games using the framework of evolutionary stochastic games proposed in [Flesch, J., Parthasarathy, T., Thuijsman, F. and Uyttendaele, P. [2013] Evolutionary stochastic games, Dyn. Games Appl. 3, 207-219]. This dynamics is defined on the simplex of all population distributions, as in classical evolutionary games. In this BR dynamics, it is shown that the set of all population distributions inducing Nash equilibria is globally asymptotically stable, for stable stochastic games. Also, if we transform the BR dynamics into the space of stationary strategies, the set of all Nash Equilibria is asymptotically stable. Another version of BR dynamics is defined in the space of stationary strategies for zero-sum stochastic games by [Leslie, D. S., Perkins, S. and Xu, Z. [2020] Best-response dynamics in zero-sum stochastic games, J. Econ. Theory 189, 105095]. To compare both versions of BR dynamics, we extend the evolutionary stochastic games model to asymmetric stochastic games and employ an equivalent definition of Nash equilibria. This comparison is illustrated using examples and it is observed that the trajectories of both versions of BR dynamics do converge to the same Nash equilibrium. The BR dynamics proposed in this paper has the advantage of being valid for nonzero sum stochastic games and conforms to both classical game dynamics and the dynamics of stationary strategies in stochastic games.