Exploratory testing: Do contextual factors influence software fault identification?

Context Exploratory Testing (ET) is a manual approach to software testing in which learning, test design and test execution occurs simultaneously. Still a developing topic of interest to academia, although as yet insufficiently investigated, most studies focus on the skills and experience of the individual tester. However, contextual factors such as project processes, test scope and organisational boundaries are also likely to affect the approach. Objective: This study explores contextual differences between teams of testers at a MedTec firm developing safety-critical products to ascertain whether contextual factors can influence the outcomes of ET, and what associated implications can be drawn for test management. Method: A development project was studied in two iterations, each consisting of a quantitative phase testing hypotheses concerning when ET would identify faults in comparison to other testing approaches and a qualitative phase involving interviews. Results: Influence on ET is traced to how the scope of tests focus learning on different types of knowledge and imply an asymmetry in the strength and number of information flows to test teams. Conclusions: While test specialisation can be attractive to software development organisations, results suggest changes to processes and organisational structures might be required to maintain test efficiency throughout projects: the responsibility for test cases might need to be rotated late in projects, and asymmetries in information flows might require management to actively strengthen the presence and connections of test teams throughout the firm. However, further research is needed to investigate whether these results also hold for non safety-critical faults.