Estimating the reproducibility of psychological science

被引：3712

作者：

Aarts, Alexander A. ^{[1
]}

Anderson, Joanna E. ^{[2
]}

Anderson, Christopher J. ^{[3
]}

Attridge, Peter R. ^{[4
,5
]}

Attwood, Angela ^{[6
]}

Axt, Jordan ^{[7
]}

Babel, Molly ^{[8
]}

Bahnik, Stepan ^{[9
]}

Baranski, Erica ^{[10
]}

Barnett-Cowan, Michael ^{[11
]}

Bartmess, Elizabeth ^{[12
]}

Beer, Jennifer ^{[13
]}

Bell, Raoul ^{[14
]}

Bentley, Heather ^{[5
]}

Beyan, Leah ^{[5
]}

Binion, Grace ^{[5
,15
]}

Borsboom, Denny ^{[16
]}

Bosch, Annick ^{[17
]}

Bosco, Frank A. ^{[18
]}

Bowman, Sara D. ^{[19
]}

Brandt, Mark J. ^{[20
]}

Braswell, Erin ^{[19
]}

Brohmer, Hilmar ^{[20
]}

Brown, Benjamin T. ^{[5
]}

Brown, Kristina ^{[5
]}

Bruening, Jovita ^{[21
,22
]}

Calhoun-Sauls, Ann ^{[23
]}

Callahan, Shannon P. ^{[24
]}

Chagnon, Elizabeth ^{[25
]}

Chandler, Jesse ^{[26
,27
]}

Chartier, Christopher R. ^{[28
]}

Cheung, Felix ^{[29
,30
]}

Christopherson, Cody D. ^{[31
]}

Cillessen, Linda ^{[17
]}

Clay, Russ ^{[32
]}

Cleary, Hayley ^{[18
]}

Cloud, Mark D. ^{[33
]}

Cohn, Michael ^{[12
]}

Cohoon, Johanna ^{[19
]}

Columbus, Simon ^{[16
]}

Cordes, Andreas ^{[34
]}

Costantini, Giulio ^{[35
]}

Alvarez, Leslie D. Cramblet ^{[36
]}

Cremata, Ed ^{[37
]}

Crusius, Jan ^{[38
]}

DeCoster, Jamie ^{[7
]}

DeGaetano, Michelle A. ^{[5
]}

Della Penna, Nicolas ^{[39
]}

den Bezemer, Bobby ^{[16
]}

Deserno, Marie K. ^{[16
]}

机构：

[1] Open Sci Collaborat, Nuenen, Netherlands

[2] Def Res & Dev Canada, Ottawa, ON, Canada

[3] Southern New Hampshire Univ, Dept Psychol, Schenectady, NY 12305 USA

[4] Mercer Sch Med, Macon, GA 31207 USA

[5] Georgia Gwinnett Coll, Lawrenceville, GA 30043 USA

[6] Univ Bristol, Sch Expt Psychol, Bristol BS8 1TH, Avon, England

[7] Univ Virginia, Charlottesville, VA 22904 USA

[8] Univ British Columbia, Vancouver, BC V6T 1Z4, Canada

[9] Univ Wurzburg, Dept Psychol 2, D-97070 Wurzburg, Germany

[10] Univ Calif Riverside, Dept Psychol, Riverside, CA 92521 USA

[11] Univ Waterloo, Waterloo, ON N2L 3G1, Canada

[12] Univ Calif San Francisco, San Francisco, CA 94118 USA

[13] Univ Texas Austin, Dept Psychol, Austin, TX 78712 USA

[14] Univ Dusseldorf, Dept Expt Psychol, Dusseldorf, Germany

[15] Univ Oregon, Dept Psychol, Eugene, OR 97403 USA

[16] Univ Amsterdam, Dept Psychol, Amsterdam, Netherlands

[17] Radboud Univ Nijmegen, NL-6525 ED Nijmegen, Netherlands

[18] Virginia Commonwealth Univ, Richmond, VA 23284 USA

[19] Ctr Open Sci, Charlottesville, VA 22902 USA

[20] Tilburg Univ, Dept Social Psychol, NL-5000 LE Tilburg, Netherlands

[21] Humboldt Univ, D-10099 Berlin, Germany

[22] Charite, D-13353 Berlin, Germany

[23] Belmont Abbey Coll, Belmont, NC 28012 USA

[24] Univ Calif Davis, Dept Psychol, Davis, CA 95616 USA

[25] Univ Maryland, Washington, DC 20012 USA

[26] Univ Michigan, Inst Social Res, Ann Arbor, MI 48104 USA

[27] Math Policy Res, Washington, DC 20002 USA

[28] Ashland Univ, Ashland, OH 44805 USA

[29] Michigan State Univ, E Lansing, MI 48824 USA

[30] Univ Hong Kong, Dept Psychol, Pok Fu Lam, Hong Kong, Peoples R China

[31] Southern Oregon Univ, Dept Psychol, Ashland, OR 97520 USA

[32] CUNY Coll Staten Isl, Staten Isl, NY 10314 USA

[33] Lock Haven Univ, Dept Psychol, Lock Haven, PA 17745 USA

[34] Univ Gottingen, D-37073 Gottingen, Germany

[35] Univ Milano Bicocca, Milan, Italy

[36] Adams State Univ, Dept Psychol, Alamosa, CO 81101 USA

[37] Univ So Calif, Los Angeles, CA 90089 USA

[38] Univ Cologne, D-50931 Cologne, Germany

[39] Australian Natl Univ, Canberra, ACT, Australia

[40] Univ Leuven, Leuven, Belgium

[41] Texas A&M Univ, College Stn, TX 77845 USA

[42] Elmhurst Coll, Elmhurst, IL 60126 USA

[43] Univ Siegen, Dept Social Psychol, D-57068 Siegen, Germany

[44] Max Planck Inst Res Collect Goods, Bonn, Germany

[45] Stockholm Sch Econ, Dept Econ, S-11383 Stockholm, Sweden

[46] Univ Kent, Sch Psychol, Canterbury, Kent, England

[47] Univ Erfurt, Erfurt, Germany

[48] Univ Washington, Seattle, WA 98195 USA

[49] Prometheus Res, New Haven, CT 06510 USA

[50] MIT, Dept Brain & Cognit Sci, Cambridge, MA 02139 USA

来源：

SCIENCE | 2015年 / 349卷 / 6251期

关键词：

REPLICABILITY; REPLICATION; PUBLICATION; INCENTIVES; PREVALENCE; TRUTH; POWER;

D O I：

10.1126/science.aac4716

中图分类号：

O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];

学科分类号：

07 ; 0710 ; 09 ;

摘要：

Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.

引用

页数：8

共 37 条

[1] An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science
Alexander, Anita
Barnett-Cowan, Michael
Bartmess, Elizabeth
Bosco, Frank A.
Brandt, Mark
Carp, Joshua
Chandler, Jesse J.
Clay, Russ
Cleary, Hayley
Cohn, Michael
Costantini, Giulio
DeCoster, Jamie
Dunn, Elizabeth
Eggleston, Casey
Estel, Vivien
Farach, Frank J.
Feather, Jenelle
Fiedler, Susann
Field, James G.
Foster, Joshua D.
Frank, Michael
Frazier, Rebecca S.
Fuchs, Heather M.
Galak, Jeff
Galliani, Elisa Maria
Garcia, Sara
Giammanco, Elise M.
Gilbert, Elizabeth A.
Giner-Sorolla, Roger
Goellner, Lars
Goh, Jin X.
Goss, R. Justin
Graham, Jesse
Grange, James A.
Gray, Jeremy R.
Gripshover, Sarah
Hartshorne, Joshua
Hayes, Timothy B.
Jahn, Georg
Johnson, Kate
Johnston, William
Joy-Gaba, Jennifer A.
Lai, Calvin K.
Lakens, Daniel
Lane, Kristin
LeBel, Etienne P.
Lee, Minha
Lemm, Kristi
Mackinnon, Sean
May, Michael
[J]. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2012, 7 (06) : 657 - 660
[2] [Anonymous], 2010, The New Yorker
[3] Raise standards for preclinical cancer research
Begley, C. Glenn
Ellis, Lee M.
[J]. NATURE, 2012, 483 (7391) : 531 - 533
[4] Continuously Cumulating Meta-Analysis and Replicability
Braver, Sanford L.
Thoemmes, Felix J.
Rosenthal, Robert
[J]. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2014, 9 (03) : 333 - 342
[5] Power failure: why small sample size undermines the reliability of neuroscience
Button, Katherine S.
Ioannidis, John P. A.
Mokrysz, Claire
Nosek, Brian A.
Flint, Jonathan
Robinson, Emma S. J.
Munafo, Marcus R.
[J]. NATURE REVIEWS NEUROSCIENCE, 2013, 14 (05) : 365 - 376
[6] STATISTICAL POWER OF ABNORMAL-SOCIAL PSYCHOLOGICAL-RESEARCH - A REVIEW
COHEN, J
[J]. JOURNAL OF ABNORMAL PSYCHOLOGY, 1962, 65 (03) : 145 - +
[7] An open investigation of the reproducibility of cancer biology research
Errington, Timothy M.
Iorns, Elizabeth
Gunn, William
Tan, Fraser Elisabeth
Lomax, Joelle
Nosek, Brian A.
[J]. ELIFE, 2014, 3
[8] "Positive" Results Increase Down the Hierarchy of the Sciences
Fanelli, Daniele
[J]. PLOS ONE, 2010, 5 (03):
[9] Theory of statistical estimation.
Fisher, RA
[J]. PROCEEDINGS OF THE CAMBRIDGE PHILOSOPHICAL SOCIETY, 1925, 22 : 700 - 725
[10] CONSEQUENCES OF PREJUDICE AGAINST NULL HYPOTHESIS
GREENWALD, AG
[J]. PSYCHOLOGICAL BULLETIN, 1975, 82 (01) : 1 - 19

← 1 2 3 4 →