Combining techniques to optimize effort predictions in software project management

This paper tackles two questions related to software effort prediction. First, is it valuable to combine prediction techniques? Second, if so, how? Many commentators have suggested the use of more than one technique in order to support effort prediction, but to date there has been little or no empirical investigation to support this recommendation. Our analysis of effort data from a medical records information system reveals that there is little, or even negative, covariance between the accuracy of our three chosen prediction techniques, namely, expert judgment, least squares regression and case-based reasoning. This indicates that when one technique predicts poorly, one or both of the others tends to perform significantly better. This is a particularly striking result given the relative homogeneity of our data set. Consequently, searching for the single "best" technique, at least in this case, leads to a suboptimal prediction strategy. The challenge then becomes one of identifying a means of determining a priori which prediction technique to use. Unfortunately, despite using a range of techniques including rule induction, we were unable to identify any simple mechanism for doing so. Nevertheless, we believe this remains an important research goal. (C) 2002 Elsevier Science Inc. All rights reserved.