Emergence of exploitation as symmetry breaking in iterated prisoner's dilemma

In society, mutual cooperation, defection, and exploitative relationships are common. Whereas cooperation and defection are studied extensively in the literature on game theory, exploitative relationships between players, in which one receives a larger benefit than the other while the game itself is symmetric, are little explored. In a recent seminal study, Press and Dyson demonstrated that if only one player can learn about the other, asymmetric exploitation is achieved in the prisoner's dilemma game. In their study, however, asymmetry is assumed in decision making between persons; the exploiting player one-sidedly determines and fixes the strategy and the exploited player follows it. It is unknown whether such exploitation emerges and is stably established even when both players learn about each other symmetrically and try to optimize their payoffs. Here, we first formulate a dynamical system that describes the change in a player's probabilistic strategy with reinforcement learning to obtain greater payoffs, based on the recognition of the other player. By applying this formulation to the standard prisoner's dilemma game, we numerically and analytically demonstrate that an exploitative relationship can be achieved despite symmetric strategy dynamics and symmetric rule of games. This exploitative relationship is stabilized by both the players: The exploiting player demands the other's unfair cooperation. Even though the exploited player, who receives a lower payoff than the exploiting player, has optimized the own strategy, the player accepts the other's defection to some degree. Whether the final equilibrium state is mutual cooperation, defection, or exploitation crucially depends on the initial conditions. Response to decrease the cooperation probability against a defector leads to oscillations in the probabilities of cooperation between the players and thus a complicated basin structure to the final equilibrium. In particular, any slight difference between both players' initial strategies can be amplified and fixed as a large difference in the probabilities of cooperation, leading to fixation of exploitation. In other words, symmetry breaking between the exploiting and exploited players results. Considering the generality of the result, this study provides another perspective on the origin of exploitation in society.