Publication bias and the canonization of false facts

被引:125
作者
Nissen, Silas Boye [1 ]
Magidson, Tali [2 ]
Gross, Kevin [3 ]
Bergstrom, Carl T. [4 ]
机构
[1] Univ Copenhagen, Niels Bohr Inst, Copenhagen, Denmark
[2] Univ Washington, Dept Comp Sci, Seattle, WA 98195 USA
[3] North Carolina State Univ, Dept Stat, Raleigh, NC USA
[4] Univ Washington, Dept Biol, Seattle, WA 98195 USA
基金
新加坡国家研究基金会;
关键词
FILE DRAWER PROBLEM; PUBLISHED RESEARCH; CLINICAL-TRIALS; INFERENCES; TESTS; NULL;
D O I
10.7554/eLife.21451
中图分类号
Q [生物科学];
学科分类号
07 ; 0710 ; 09 ;
摘要
Science is facing a replication crisis in which many experimental findings cannot be replicated and are likely to be false. Does this imply that many scientific facts are false as well? To find out, we explore the process by which a claim becomes fact. We model the communitys confidence in a claim as a Markov process with successive published results shifting the degree of belief. Publication bias in favor of positive findings influences the distribution of published results. We find that unless a sufficient fraction of negative results are published, false claims frequently can become canonized as fact. Data-dredging, p-hacking, and similar behaviors exacerbate the problem. Should negative results become easier to publish as a claim approaches acceptance as a fact, however, true and false claims would be more readily distinguished. To the degree that the model reflects the real world, there may be serious concerns about the validity of purported facts in some disciplines.
引用
收藏
页数:19
相关论文
共 55 条
[1]   Estimating the reproducibility of psychological science [J].
Aarts, Alexander A. ;
Anderson, Joanna E. ;
Anderson, Christopher J. ;
Attridge, Peter R. ;
Attwood, Angela ;
Axt, Jordan ;
Babel, Molly ;
Bahnik, Stepan ;
Baranski, Erica ;
Barnett-Cowan, Michael ;
Bartmess, Elizabeth ;
Beer, Jennifer ;
Bell, Raoul ;
Bentley, Heather ;
Beyan, Leah ;
Binion, Grace ;
Borsboom, Denny ;
Bosch, Annick ;
Bosco, Frank A. ;
Bowman, Sara D. ;
Brandt, Mark J. ;
Braswell, Erin ;
Brohmer, Hilmar ;
Brown, Benjamin T. ;
Brown, Kristina ;
Bruening, Jovita ;
Calhoun-Sauls, Ann ;
Callahan, Shannon P. ;
Chagnon, Elizabeth ;
Chandler, Jesse ;
Chartier, Christopher R. ;
Cheung, Felix ;
Christopherson, Cody D. ;
Cillessen, Linda ;
Clay, Russ ;
Cleary, Hayley ;
Cloud, Mark D. ;
Cohn, Michael ;
Cohoon, Johanna ;
Columbus, Simon ;
Cordes, Andreas ;
Costantini, Giulio ;
Alvarez, Leslie D. Cramblet ;
Cremata, Ed ;
Crusius, Jan ;
DeCoster, Jamie ;
DeGaetano, Michelle A. ;
Della Penna, Nicolas ;
den Bezemer, Bobby ;
Deserno, Marie K. .
SCIENCE, 2015, 349 (6251)
[2]   Go forth and replicate! [J].
不详 .
NATURE, 2016, 536 (7617) :373-373
[3]  
[Anonymous], 2012, The Half-life of Facts: Why Everything we Know has an Expiration Date
[4]  
Baker M, 2016, NATURE, V533, P452, DOI 10.1038/533452a
[5]   PUBLICATION BIAS - A PROBLEM IN INTERPRETING MEDICAL DATA [J].
BEGG, CB ;
BERLIN, JA .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES A-STATISTICS IN SOCIETY, 1988, 151 :419-463
[6]   Raise standards for preclinical cancer research [J].
Begley, C. Glenn ;
Ellis, Lee M. .
NATURE, 2012, 483 (7391) :531-533
[7]   Role for a bidentate ribonuclease in the initiation step of RNA interference [J].
Bernstein, E ;
Caudy, AA ;
Hammond, SM ;
Hannon, GJ .
NATURE, 2001, 409 (6818) :363-366
[8]   Evaluating replicability of laboratory experiments in economics [J].
Camerer, Colin F. ;
Dreber, Anna ;
Forsell, Eskil ;
Ho, Teck-Hua ;
Huber, Juergen ;
Johannesson, Magnus ;
Kirchler, Michael ;
Almenberg, Johan ;
Altmejd, Adam ;
Chan, Taizan ;
Heikensten, Emma ;
Holzmeister, Felix ;
Imai, Taisuke ;
Isaksson, Siri ;
Nave, Gideon ;
Pfeiffer, Thomas ;
Razen, Michael ;
Wu, Hang .
SCIENCE, 2016, 351 (6280) :1433-1436
[9]   Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors [J].
Chan, AW ;
Altman, DG .
BMJ-BRITISH MEDICAL JOURNAL, 2005, 330 (7494) :753-756
[10]  
Chang A.C., 2015, FINANCE EC DISCUSSIO, DOI [DOI 10.17016/FEDS.2015.083, 10.17016/FEDS.2015.083]