Improving the Replicability of Psychological Science Through Pedagogy

被引:27
作者
Hawkins, Robert X. D. [1 ]
Smith, Eric N. [1 ]
Au, Carolyn [1 ]
Arias, Juan Miguel [1 ]
Catapano, Rhia [1 ]
Hermann, Eric [1 ]
Keil, Martin [1 ]
Lampinen, Andrew [1 ]
Raposo, Sarah [1 ]
Reynolds, Jesse [1 ]
Salehi, Shima [1 ]
Salloum, Justin [1 ]
Tan, Jed [1 ]
Frank, Michael C. [1 ]
机构
[1] Stanford Univ, Dept Psychol, Stanford, CA 94305 USA
基金
美国国家科学基金会;
关键词
replication; reproducibility; pedagogy; experimental methods; open data; open materials; preregistered; MECHANICAL TURK; REPRODUCIBILITY; REPLICATIONS; ASSOCIATION; SMARTPHONE;
D O I
10.1177/2515245917740427
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
Replications are important to science, but who will do them? One proposal is that students can conduct replications as part of their training. As a proof of concept for this idea, here we report a series of 11 preregistered replications of findings from the 2015 volume of Psychological Science, all conducted as part of a graduate-level course. As was expected given larger, more systematic prior efforts, the replications typically yielded effects that were smaller than the original ones: The modal outcome was partial support for the original claim. This work documents the challenges facing motivated students as they attempt to replicate previously published results on a first attempt. We describe the workflow and pedagogical methods that were used in the class and discuss implications both for the adoption of this pedagogical model and for replication research more broadly.
引用
收藏
页码:7 / 18
页数:12
相关论文
共 47 条
  • [1] Estimating the reproducibility of psychological science
    Aarts, Alexander A.
    Anderson, Joanna E.
    Anderson, Christopher J.
    Attridge, Peter R.
    Attwood, Angela
    Axt, Jordan
    Babel, Molly
    Bahnik, Stepan
    Baranski, Erica
    Barnett-Cowan, Michael
    Bartmess, Elizabeth
    Beer, Jennifer
    Bell, Raoul
    Bentley, Heather
    Beyan, Leah
    Binion, Grace
    Borsboom, Denny
    Bosch, Annick
    Bosco, Frank A.
    Bowman, Sara D.
    Brandt, Mark J.
    Braswell, Erin
    Brohmer, Hilmar
    Brown, Benjamin T.
    Brown, Kristina
    Bruening, Jovita
    Calhoun-Sauls, Ann
    Callahan, Shannon P.
    Chagnon, Elizabeth
    Chandler, Jesse
    Chartier, Christopher R.
    Cheung, Felix
    Christopherson, Cody D.
    Cillessen, Linda
    Clay, Russ
    Cleary, Hayley
    Cloud, Mark D.
    Cohn, Michael
    Cohoon, Johanna
    Columbus, Simon
    Cordes, Andreas
    Costantini, Giulio
    Alvarez, Leslie D. Cramblet
    Cremata, Ed
    Crusius, Jan
    DeCoster, Jamie
    DeGaetano, Michelle A.
    Della Penna, Nicolas
    den Bezemer, Bobby
    Deserno, Marie K.
    [J]. SCIENCE, 2015, 349 (6251)
  • [2] Response to Comment on "Estimating the reproducibility of psychological science"
    Anderson, Christopher J.
    Bahnik, Stepan
    Barnett-Cowan, Michael
    Bosco, Frank A.
    Chandler, Jesse
    Chartier, Christopher R.
    Cheung, Felix
    Christopherson, Cody D.
    Cordes, Andreas
    Cremata, Edward J.
    Della Penna, Nicolas
    Estel, Vivien
    Fedor, Anna
    Fitneva, Stanka A.
    Frank, Michael C.
    Grange, James A.
    Hartshorne, Joshua K.
    Hasselman, Fred
    Henninger, Felix
    van der Hulst, Marije
    Jonas, Kai J.
    Lai, Calvin K.
    Levitan, Carmel A.
    Miller, Jeremy K.
    Moore, Katherine S.
    Meixner, Johannes M.
    Munafo, Marcus R.
    Neijenhuijs, Koen I.
    Nilsonne, Gustav
    Nosek, Brian A.
    Plessow, Franziska
    Prenoveau, Jason M.
    Ricker, Ashley A.
    Schmidt, Kathleen
    Spies, Jeffrey R.
    Stieger, Stefan
    Strohminger, Nina
    Sullivan, Gavin B.
    van Aert, Robbie C. M.
    van Assen, Marcel A. L. M.
    Vanpaemel, Wolf
    Vianello, Michelangelo
    Voracek, Martin
    Zuni, Kellylynn
    [J]. SCIENCE, 2016, 351 (6277)
  • [3] [Anonymous], 2016, ESSAYS XCELLENCE TEA
  • [4] When Knowledge Knows No Bounds: Self-Perceived Expertise Predicts Claims of Impossible Knowledge
    Atir, Stav
    Rosenzweig, Emily
    Dunning, David
    [J]. PSYCHOLOGICAL SCIENCE, 2015, 26 (08) : 1295 - 1303
  • [6] Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
    Buhrmester, Michael
    Kwang, Tracy
    Gosling, Samuel D.
    [J]. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE, 2011, 6 (01) : 3 - 5
  • [7] Power failure: why small sample size undermines the reliability of neuroscience
    Button, Katherine S.
    Ioannidis, John P. A.
    Mokrysz, Claire
    Nosek, Brian A.
    Flint, Jonathan
    Robinson, Emma S. J.
    Munafo, Marcus R.
    [J]. NATURE REVIEWS NEUROSCIENCE, 2013, 14 (05) : 365 - 376
  • [8] Using Nonnaive Participants Can Reduce Effect Sizes
    Chandler, Jesse
    Paolacci, Gabriele
    Peer, Eyal
    Mueller, Pam
    Ratliff, Kate A.
    [J]. PSYCHOLOGICAL SCIENCE, 2015, 26 (07) : 1131 - 1139
  • [9] Nonnaivete among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers
    Chandler, Jesse
    Mueller, Pam
    Paolacci, Gabriele
    [J]. BEHAVIOR RESEARCH METHODS, 2014, 46 (01) : 112 - 130
  • [10] Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research
    Crump, Matthew J. C.
    McDonnell, John V.
    Gureckis, Todd M.
    [J]. PLOS ONE, 2013, 8 (03):