Empirical software product line engineering: A systematic literature review. An IST journal publication

The adoption of Software Product Line Engineering (SPLE) is usually only based on its theoretical benefits instead of empirical evidences. In fact, there is no work that synthesizes the empirical studies on SPLE. This makes it difficult for researchers to base their contributions on previous works validated with an empirical strategy. The objective of this work is to discover and summarize the studies that have used empirical evidences in SPLE limited to those ones with the intervention of humans. This will allow evaluating the quality and knowing the scope of these studies over time. Doing so, research opportunities can arise. Analyzing the authors and institutions that investigate SPLE supported by empirical studies will also help to know which institutions have knowledge of the subject, leading to detect and encourage collaboration among researches. A systematic literature review was conducted with the focus on those studies in which there is human intervention and were published between 2000 and 2018 (the systematic literature review was developed in 2019). We considered peer-reviewed papers from journals and top software engineering conferences. Out of a total of 1880 studies in the initial set, a total of 62 primary studies were selected after applying a series of inclusion and exclusion criteria. We found that, approximately 56% of the studies used the empirical case study strategy while the rest used experimental strategies. Around 86% of the case studies were performed in an industrial environment showing the penetration of SPLE in industry while 81% of the experiments were conducted in an academic environment. Around 95.16% of the studies address aspects related to domain engineering while application engineering received less attention. Most of the experiments and case study evaluated showed an acceptable level of quality. The first study found dates from 2005 and since 2008, the interest in the empirical SPLE has increased.