Competing Bandits: The Perils of Exploration Under Competition

Most online platforms learn from interactions with users and engage in exploration: making potentially suboptimal choices to acquire new information. We study the interplay between exploration and competition: how such platforms balance the exploration for learning and competition for users. We consider a stylized duopoly in which two firms face the same multi-armed bandit problem. Users arrive one by one and choose between the two firms, so that each firm makes progress on its bandit problem only if it is chosen. We study whether competition incentivizes the adoption of better algorithms. We find that stark competition disincentivizes exploration, leading to low welfare. However, weaker competition incentivizes better exploration algorithms and increases welfare. We investigate two channels for weakening the competition: stochastic user choice models and a first-mover advantage. Our findings speak to the competition-innovation relationship and the first-mover advantage in the digital economy.