How Many Web APIs Evolve Following Semantic Versioning?

More and more Web APIs use semantic versioning to represent the impact of changes on clients depending on previous versions. Our goal is to provide insights about the extent to which evolving Web APIs align with semantic versioning rules. In this paper we present the results of an empirical study on the descriptions of 3 075Web APIs, which released at least one new version throughout their history. The APIs descriptions were mined by retrieving 132 909 commits from 2 028 different open source GitHub repositories. We systematically collected and examined 506 273 changes of 195 different types released within 16 053 new API versions. We classified whether each change is likely to break clients or not, and checked whether the corresponding version identifier has been updated following semantic versioning rules. The results indicate that in the best case, only 517 APIs consistently release major upgrades when introducing breaking changes, while 1 970 APIs will not always correctly inform their clients about breaking changes released as part of minor or patch-level upgrades. We also detected 927 APIs which use a backwards-compatible evolution strategy, as they never introduce any breaking change throughout their history.