Many Labs 3: Evaluating participant pool quality across the academic semester via replication

被引：230

作者：

Ebersole, Charles R. ^{[1
]}

Atherton, Olivia E. ^{[2
]}

Belanger, Aimee L. ^{[3
]}

Skulborstad, Hayley M. ^{[3
]}

Allen, Jill M. ^{[4
]}

Banks, Jonathan B. ^{[5
]}

Baranski, Erica ^{[6
]}

Bernstein, Michael J. ^{[7
]}

Bonfiglio, Diane B. V. ^{[8
]}

Boucher, Leanne ^{[5
]}

Brown, Elizabeth R. ^{[9
]}

Budiman, Nancy I. ^{[2
]}

Cairo, Athena H. ^{[10
]}

Capaldi, Colin A. ^{[11
]}

Chartier, Christopher R. ^{[8
]}

Chung, Joanne M. ^{[2
]}

Cicero, David C. ^{[12
]}

Coleman, Jennifer A. ^{[10
]}

Conway, John G. ^{[13
]}

Davis, William E. ^{[14
]}

Devos, Thierry ^{[15
]}

Fletcher, Melody M. ^{[16
]}

German, Komi ^{[2
]}

Grahe, Jon E. ^{[17
]}

Hermann, Anthony D. ^{[18
]}

Hicks, Joshua A. ^{[14
]}

Honeycutt, Nathan ^{[15
]}

Humphrey, Brandon ^{[3
]}

Janus, Matthew ^{[1
]}

Johnson, David J. ^{[19
]}

Joy-Gaba, Jennifer A. ^{[10
]}

Juzeler, Hannah ^{[17
]}

Keres, Ashley ^{[8
]}

Kinney, Diana ^{[1
]}

Kirshenbaum, Jacqeline ^{[18
]}

Klein, Richard A. ^{[13
]}

Lucas, Richard E. ^{[19
]}

Lustgraaf, Christopher J. N. ^{[16
]}

Martin, Daniel ^{[1
]}

Menon, Madhavi ^{[5
]}

Metzger, Mitchell ^{[8
]}

Moloney, Jaclyn M. ^{[10
]}

Morse, Patrick J. ^{[6
]}

Prislin, Radmila ^{[15
]}

Razza, Timothy ^{[5
]}

Re, Daniel E.

Rule, Nicholas O. ^{[20
]}

Sacco, Tdonald F. ^{[16
]}

Sauerberger, Kyle ^{[6
]}

Shrider, Emily ^{[8
]}

机构：

[1] Univ Virginia, Charlottesville, VA 22904 USA

[2] Univ Calif Davis, Davis, CA USA

[3] Miami Univ, Oxford, OH 45056 USA

[4] Montana State Univ, Bozeman, MT USA

[5] Nova SE Univ, Ft Lauderdale, FL 33314 USA

[6] Univ Calif Riverside, Riverside, CA USA

[7] Penn State Univ, Abington, PA USA

[8] Ashland Univ, Ashland, OH USA

[9] Univ N Florida, Jacksonville, FL 32224 USA

[10] Virginia Commonwealth Univ, Richmond, VA 23284 USA

[11] Carleton Univ, Ottawa, ON K1S 5B6, Canada

[12] Univ Hawaii Manoa, Honolulu, HI 96822 USA

[13] Univ Florida, Gainesville, FL 32611 USA

[14] Texas A&M Univ, College Stn, TX 77843 USA

[15] San Diego State Univ, San Diego, CA 92182 USA

[16] Univ So Mississippi, Hattiesburg, MS 39406 USA

[17] Pacific Lutheran Univ, Tacoma, WA 98447 USA

[18] Bradley Univ, Peoria, IL 61625 USA

[19] Michigan State Univ, E Lansing, MI 48824 USA

[20] Univ Toronto, Toronto, ON M5S 1A1, Canada

[21] Ithaca Coll, Ithaca, NY USA

[22] Ohio State Univ Newark, Newark, OH USA

[23] Univ Virginia, Ctr Open Sci, Charlottesville, VA 22903 USA

来源：

JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY | 2016年 / 67卷

关键词：

Social psychology; Cognitive psychology; Replication; Participant pool; Individual differences; Sampling effects; Situational effects; SELF-ESTEEM; PERSONALITY; NEED; TIME; INFORMATION; JUDGMENTS; ATTENTION; COGNITION;

D O I：

10.1016/j.jesp.2015.10.012

中图分类号：

B84 [心理学];

学科分类号：

04 ; 0402 ;

摘要：

The university participant pool is a key resource for behavioral research, and data quality is believed to vary over the course of the academic semester. This crowdsourced project examined time of semester variation in 10 known effects, 10 individual differences, and 3 data quality indicators over the course of the academic semester in 20 participant pools (N = 2696) and with an online sample (N = 737). Weak time of semester effects were observed on data quality indicators, participant sex, and a few individual differences conscientiousness, mood, and stress. However, there was little evidence for time of semester qualifying experimental or correlational effects. The generality of this evidence is unknown because only a subset of the tested effects demonstrated evidence for the original result in the whole sample. Mean characteristics of pool samples change slightly during the semester, but these data suggest that those changes are mostly irrelevant for detecting effects. (C) 2015 Elsevier Inc. All rights reserved.

引用

页码：68 / 82

页数：15

共 48 条

[1] Estimating the reproducibility of psychological science [J].

Aarts, Alexander A. ;

Anderson, Joanna E. ;

Anderson, Christopher J. ;

Attridge, Peter R. ;

Attwood, Angela ;

Axt, Jordan ;

Babel, Molly ;

Bahnik, Stepan ;

Baranski, Erica ;

Barnett-Cowan, Michael ;

Bartmess, Elizabeth ;

Beer, Jennifer ;

Bell, Raoul ;

Bentley, Heather ;

Beyan, Leah ;

Binion, Grace ;

Borsboom, Denny ;

Bosch, Annick ;

Bosco, Frank A. ;

Bowman, Sara D. ;

Brandt, Mark J. ;

Braswell, Erin ;

Brohmer, Hilmar ;

Brown, Benjamin T. ;

Brown, Kristina ;

Bruening, Jovita ;

Calhoun-Sauls, Ann ;

Callahan, Shannon P. ;

Chagnon, Elizabeth ;

Chandler, Jesse ;

Chartier, Christopher R. ;

Cheung, Felix ;

Christopherson, Cody D. ;

Cillessen, Linda ;

Clay, Russ ;

Cleary, Hayley ;

Cloud, Mark D. ;

Cohn, Michael ;

Cohoon, Johanna ;

Columbus, Simon ;

Cordes, Andreas ;

Costantini, Giulio ;

Alvarez, Leslie D. Cramblet ;

Cremata, Ed ;

Crusius, Jan ;

DeCoster, Jamie ;

DeGaetano, Michelle A. ;

Della Penna, Nicolas ;

den Bezemer, Bobby ;

Deserno, Marie K. .

SCIENCE, 2015, 349 (6251)

[2] THE WORK PREFERENCE INVENTORY - ASSESSING INTRINSIC AND EXTRINSIC MOTIVATIONAL ORIENTATIONS [J].

AMABILE, TM ;

HILL, KG ;

HENNESSEY, BA ;

TIGHE, EM .

JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY, 1994, 66 (05) :950-967

[3]

[Anonymous], MANY LABS UNPUB

[4]

[Anonymous], 2012, EFFECT SIZES RES UNI

[5] Optimism and self-mastery predict more rapid disengagement from unsolvable tasks in the presence of alternatives [J].

Aspinwall, LG ;

Richter, L .

MOTIVATION AND EMOTION, 1999, 23 (03) :221-245

[6] Who comes when: personality differences in early and later participation in a university subject pool [J].

Aviv, AL ;

Zelenski, JM ;

Rallo, L ;

Larsen, RJ .

PERSONALITY AND INDIVIDUAL DIFFERENCES, 2002, 33 (03) :487-496

[7]

Boroditsky L, 2000, COGNITION, V75, P1, DOI 10.1016/S0010-0277(99)00073-6

[8] THE NEED FOR COGNITION [J].

CACIOPPO, JT ;

PETTY, RE .

JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY, 1982, 42 (01) :116-131

[9] THE EFFICIENT ASSESSMENT OF NEED FOR COGNITION [J].

CACIOPPO, JT ;

PETTY, RE ;

KAO, CF .

JOURNAL OF PERSONALITY ASSESSMENT, 1984, 48 (03) :306-307

[10] EFFECTS OF NEED FOR COGNITION ON MESSAGE EVALUATION, RECALL, AND PERSUASION [J].

CACIOPPO, JT ;

PETTY, RE ;

MORRIS, KJ .

JOURNAL OF PERSONALITY AND SOCIAL PSYCHOLOGY, 1983, 45 (04) :805-818

← 1 2 3 4 5 →