Maintenance Effort Estimation for Open Source Software: A Systematic Literature Review

Open Source Software (OSS) is distributed and maintained collaboratively by developers all over the world. However, frequent personnel turnover and lack of organizational management makes it difficult to capture the actual development effort. Various OSS maintenance effort estimation approaches have been developed to provide a way to understand and estimate development effort. The goal of this study is to identify the current state of art of the existing maintenance effort estimation approaches for OSS. We performed a systematic literature review on the relevant studies published in the period between 2000-2015 by both automatic and manual searches from different sources. We derived a set of keywords from the research questions and established selection criteria to carefully choose the papers to evaluate. 29 out of 3,312 papers were selected based on a well designed selection process. Our results show that the commonly used OSS maintenance effort estimation methods are actual effort estimation and maintenance activity time prediction; the most commonly used metrics and factors for actual effort estimation are source code measurements and people related metrics; the most commonly mentioned activity for maintenance activity time prediction is bug fixing. Accuracy measures and cross validation is used for validating the estimation models. Based on the above findings, we identified the issues in evaluation methods for actual maintenance effort estimations and the needs for quantitative OSS maintenance effort inference from size-related metrics. Meanwhile, we highlighted individual contribution and performance measurement as a novel and promising research area.