On the Difficulty of Replicating Human Subjects Studies in Software Engineering

被引：0

作者：

Lung, Jonathan ^{[1
]}

Aranda, Jorge ^{[1
]}

Easterbrook, Steve ^{[1
]}

Wilson, Greg ^{[1
]}

机构：

[1] Univ Toronto, Dept Comp Sci, Toronto, ON M5S 2E4, Canada

来源：

ICSE'08 PROCEEDINGS OF THE THIRTIETH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING | 2008年

关键词：

experience report; empirical; human subjects; replication;

D O I：

暂无

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

Replications play an important role in verifying empirical results. In this paper, we discuss our experiences performing a literal replication of a human subjects experiment that examined the relationship between a simple test for consistent use of mental models, and success in an introductory programming course. We encountered many difficulties in achieving comparability with the original experiment, due to a series of apparently minor differences in context. Based on this experience, we discuss the relative merits of replication, and suggest that, for some human subjects studies, literal replication may not be the the most effective strategy for validating the results of previous studies.

引用

页码：191 / 200

页数：10

共 15 条

[1]

[Anonymous], 1995, READINGS HUMAN COMPU

[2] Building knowledge through families of experiments [J].

Basili, VR ;

Lanubile, F .

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 1999, 25 (04) :456-473

[3]

Brooks A, 2007, GUIDE ADV EMPIRICAL, P365

[4]

CAMPBELL J, 2006, CSC108H1S FINAL EXAM

[5]

CASPERSEN ME, 2007, ITICSE, V39, P206

[6]

Dehnadi S., 2006, CAMEL HAS 2 HUMPS

[7]

DEHNADI S, 2007, COMMUNICATION

[8]

Easterbrook S., 2008, Guide to advanced empirical software engineering, P285, DOI [10.1007/978-1-84800-044-5_11, DOI 10.1007/978-1-84800-044-5_11]

[9]

JORGENSEN M, 2004, EMPIRICAL ASSESSMENT, P29

[10]

Kitchenham BA, 2004, PROC INT CONF SOFTW, P273

← 1 2 →