A Systematic Mapping Study on Software Engineering Testbeds

Even though empirical research has grown in interest, techniques, methodologies and best practices are still in debate. In this context, testbeds are effective when one needs to evaluate and compare technologies. The concept is well disseminated in other areas such as Computer Networks, but remains poorly explored in Software Engineering (SE). This paper presents a systematic mapping study on the SE testbeds literature. From the initial set of 4239 studies, 13 primary studies were selected and categorized. Based on that, we found that Software Architecture is the most investigated topic, controlled experiment is the most used method to evaluate such testbeds, 20 benefits of using testbeds in SE have been identified and that testbeds comprise very heterogeneous structural elements.