Replicating software engineering experiments: a poisoned chalice or the Holy Grail

Recently, software engineering has witnessed a great increase in the amount of work with an empirical component; however, this work has often little or no established empirical framework within the topic to draw upon. Frequently, researchers use frameworks from other disciplines in an attempt to alleviate this deficiency. A common underpinning in these frameworks is that experimental replication is available as the cornerstone of knowledge discovery within the discipline. This paper investigates the issues involved in accepting this premise as a fundamental building block with empirical software engineering and recommends extending the traditional view of replication to improve the effectiveness of this essential process within our domain. (C) 2004 Elsevier B.V. All rights reserved.