Behavioral flexibility is measured using a plethora of non-equivalent tasks. In experimental psychology, tests include reversal learning, set-shifting and self-control tasks. In addition to the latter tasks, behavioral ecologists have measured behavioral flexibility with tasks that include problem-solving and personality tests. A relationship between behavioral flexibility assessments, especially in behavioral ecology, still needs to be demonstrated. We argue that using a single umbrella term that incorporates too many unrelated traits should be avoided.Behavioral ecologists interested in comparative cognition have struggled to design tasks that are both ecologically relevant and experimentally rigorous. In experimental psychology, standardized tests of reversal learning, set-shifting and self-control have long been used to measure aspects of flexible behavior especially with regards to determining the neural mechanisms that enable animals and humans to rapidly and efficiently adapt to different situations. More recently, behavioral ecologists have used the term "behavioral flexibility" more broadly to explain differences in traits such as personality and innovation. Here, we argue that the term behavioral flexibility designates too many non-equivalent traits, and that this can lead to misconceptions about the nature of cognitive abilities.