Developmental psychologists should adopt citizen science to improve generalization and reproducibility

被引:10
作者
Li, Wei [1 ]
Germine, Laura Thi [2 ,3 ]
Mehr, Samuel A. [4 ,5 ]
Srinivasan, Mahesh [6 ]
Hartshorne, Joshua [1 ]
机构
[1] Boston Coll, Dept Psychol & Neurosci, 140 Commonwealth Ave, Chestnut Hill, MA 02467 USA
[2] McLean Hosp, 115 Mill St, Belmont, MA 02178 USA
[3] Harvard Med Sch, Dept Psychiat, Cambridge, MA USA
[4] Harvard Univ, Data Sci Initiat, Cambridge, MA 02138 USA
[5] Victoria Univ Wellington, Sch Psychol, Wellington, New Zealand
[6] Univ Calif Berkeley, Dept Psychol, 3210 Tolman Hall, Berkeley, CA 94720 USA
基金
美国国家科学基金会; 美国国家卫生研究院;
关键词
citizen science; generalizability; online experiments; replicability; STATISTICAL POWER; MECHANICAL TURK; SOCIAL-PSYCHOLOGY; CRITICAL PERIOD; EFFECT SIZES; SAMPLE-SIZE; LIFE-SPAN; PERSONALITY; LANGUAGE; FAILURE;
D O I
10.1002/icd.2348
中图分类号
B844 [发展心理学(人类心理学)];
学科分类号
040202 ;
摘要
Widespread failures of replication and generalization are, ironically, a scientific triumph, in that they confirm the fundamental metascientific theory that underlies our field. Generalizable and replicable findings require testing large numbers of subjects from a wide range of demographics with a large, randomly-sampled stimulus set, and using a variety of experimental parameters. Because few studies accomplish any of this, meta-scientists predict that findings will frequently fail to replicate or generalize. We argue that to be more robust and replicable, developmental psychology needs to find a mechanism for collecting data at a greater scale and from more diverse populations. Luckily, this mechanism already exists as follows: Citizen science, in which large numbers of uncompensated volunteers provide data. While best-known for its contributions to astronomy and ecology, citizen science has also produced major findings in neuroscience and psychology, and increasingly in developmental psychology. We provide examples, address practical challenges, discuss limitations, and compare to other methods of obtaining large datasets. Ultimately, we argue that the range of studies where it makes sense *not* to use citizen science is steadily dwindling.
引用
收藏
页数:23
相关论文
共 209 条
[1]   Estimating the reproducibility of psychological science [J].
Aarts, Alexander A. ;
Anderson, Joanna E. ;
Anderson, Christopher J. ;
Attridge, Peter R. ;
Attwood, Angela ;
Axt, Jordan ;
Babel, Molly ;
Bahnik, Stepan ;
Baranski, Erica ;
Barnett-Cowan, Michael ;
Bartmess, Elizabeth ;
Beer, Jennifer ;
Bell, Raoul ;
Bentley, Heather ;
Beyan, Leah ;
Binion, Grace ;
Borsboom, Denny ;
Bosch, Annick ;
Bosco, Frank A. ;
Bowman, Sara D. ;
Brandt, Mark J. ;
Braswell, Erin ;
Brohmer, Hilmar ;
Brown, Benjamin T. ;
Brown, Kristina ;
Bruening, Jovita ;
Calhoun-Sauls, Ann ;
Callahan, Shannon P. ;
Chagnon, Elizabeth ;
Chandler, Jesse ;
Chartier, Christopher R. ;
Cheung, Felix ;
Christopherson, Cody D. ;
Cillessen, Linda ;
Clay, Russ ;
Cleary, Hayley ;
Cloud, Mark D. ;
Cohn, Michael ;
Cohoon, Johanna ;
Columbus, Simon ;
Cordes, Andreas ;
Costantini, Giulio ;
Alvarez, Leslie D. Cramblet ;
Cremata, Ed ;
Crusius, Jan ;
DeCoster, Jamie ;
DeGaetano, Michelle A. ;
Della Penna, Nicolas ;
den Bezemer, Bobby ;
Deserno, Marie K. .
SCIENCE, 2015, 349 (6251)
[2]  
Addyman C., 2013, Comedy Studies, V4, P143, DOI [https://doi.org/10.1386/cost.4.2.1431, DOI 10.1386/COST.4.2.1431, 10.1386/cost.4.2.143_1, DOI 10.1386/COST.4.2.143_1]
[3]  
[Anonymous], 2014, PRIVACY SENSOR DRIVE
[4]  
[Anonymous], 2020, How many children and young people have internet access at home? Estimating digital connectivity during the COVID-19 pandemic
[5]  
[Anonymous], 2000, American's Kindergartners: Findings from the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99, Fall 1998
[6]  
Ariel J., 2022, WHAT PARADIGMS CAN W
[7]  
Auxier B., 2020, PARENTING CHILDREN A
[8]   The Moral Machine experiment [J].
Awad, Edmond ;
Dsouza, Sohan ;
Kim, Richard ;
Schulz, Jonathan ;
Henrich, Joseph ;
Shariff, Azim ;
Bonnefon, Jean-Francois ;
Rahwan, Iyad .
NATURE, 2018, 563 (7729) :59-+
[9]   Mixed-effects modeling with crossed random effects for subjects and items [J].
Baayen, R. H. ;
Davidson, D. J. ;
Bates, D. M. .
JOURNAL OF MEMORY AND LANGUAGE, 2008, 59 (04) :390-412
[10]   Validation of the Emotiv EPOC® EEG gaming systemfor measuring research quality auditory ERPs [J].
Badcock, Nicholas A. ;
Mousikou, Petroula ;
Mahajan, Yatin ;
de Lissa, Peter ;
Thie, Johnson ;
McArthur, Genevieve .
PEERJ, 2013, 1