Manipulating the Alpha Level Cannot Cure Significance Testing

被引：60

作者：

Trafimow, David ^{[1
]}

Amrhein, Valentin ^{[2
,3
]}

Areshenkoff, Corson N. ^{[4
]}

Barrera-Causil, Carlos J. ^{[5
]}

Beh, Eric J. ^{[6
]}

Bilgic, Yusuf K. ^{[7
]}

Bono, Roser ^{[8
,9
]}

Bradley, Michael T. ^{[10
]}

Briggs, William M.

Cepeda-Freyre, Hector A. ^{[11
]}

Chaigneau, Sergio E. ^{[12
]}

Ciocca, Daniel R. ^{[13
]}

Correa, Juan C. ^{[14
]}

Cousineau, Denis ^{[15
]}

de Boer, Michiel R. ^{[16
,17
]}

Dhar, Subhra S. ^{[18
]}

Dolgov, Igor ^{[1
]}

Gomez-Benito, Juana ^{[8
,9
]}

Grendar, Marian ^{[19
,20
]}

Grice, James W. ^{[21
]}

Guerrero-Gimenez, Martin E. ^{[13
]}

Gutierrez, Andres ^{[22
]}

Huedo-Medina, Tania B. ^{[23
]}

Jaffe, Klaus ^{[24
]}

Janyan, Armina ^{[25
,26
]}

Karimnezhad, Ali ^{[27
]}

Korner-Nievergelt, Franzi ^{[3
,28
]}

Kosugi, Koji ^{[29
]}

Lachmair, Martin ^{[30
]}

Ledesma, Ruben D. ^{[31
,32
]}

Limongi, Roberto ^{[33
,34
]}

Liuzza, Marco T. ^{[35
]}

Lombardo, Rosaria ^{[36
]}

Marks, Michael J. ^{[1
]}

Meinlschmidt, Gunther ^{[37
,38
,39
,40
]}

Nalborczyk, Ladislas ^{[41
,42
]}

Nguyen, Hung T. ^{[43
]}

Ospina, Raydonal ^{[44
]}

Perezgonzalez, Jose D. ^{[45
]}

Pfister, Roland ^{[46
]}

Rahona, Juan J. ^{[30
]}

Rodriguez-Medina, David A. ^{[47
]}

Romao, Xavier ^{[48
]}

Ruiz-Fernandez, Susana ^{[30
,49
,50
,51
]}

Suarez, Isabel ^{[52
]}

Tegethoff, Marion ^{[53
]}

Tejo, Mauricio ^{[54
]}

van de Schoot, Rens ^{[55
,56
]}

Vankov, Ivan I. ^{[25
]}

Velasco-Forero, Santiago ^{[57
]}

机构：

[1] New Mexico State Univ, Dept Psychol, Las Cruces, NM 88003 USA

[2] Univ Basel, Zool Inst, Basel, Switzerland

[3] Swiss Ornithol Inst, Sempach, Switzerland

[4] Queens Univ, Ctr Neurosci Studies, Kingston, ON, Canada

[5] Fac Appl & Exact Sci, Metropolitan Technol Inst, Medellin, Colombia

[6] Univ Newcastle, Sch Math & Phys Sci, Callaghan, NSW, Australia

[7] SUNY Coll Geneseo, Dept Math, Geneseo, NY 14454 USA

[8] Univ Barcelona, Fac Psychol, Quantitat Psychol Unit, Barcelona, Spain

[9] Univ Barcelona, Inst Neurociencies, Barcelona, Spain

[10] Univ New Brunswick, Dept Psychol, St John, NB, Canada

[11] Benemerita Univ Autonoma Puebla, Sch Psychol, Puebla, Mexico

[12] Univ Adolfo Ibanez, Sch Psychol, Ctr Social & Cognit Neurosci, Santiago, Chile

[13] CONICET Mendoza, CCT, Inst Med Biol Expt Cuyo, Oncol Lab, Mendoza, Argentina

[14] Univ Nacl Colombia, Fac Sci, Sch Stat, Medellin, Colombia

[15] Univ Ottawa, Sch Psychol, Ottawa, ON, Canada

[16] Vrije Univ Amsterdam, Dept Hlth Sci, Amsterdam, Netherlands

[17] Amsterdam Publ Hlth Res Inst, Amsterdam, Netherlands

[18] Indian Inst Technol, Dept Math & Stat, Kanpur, Uttar Pradesh, India

[19] Comenius Univ, Jessenius Fac Med, Biomed Ctr Martin, Martin, Slovakia

[20] Slovak Acad Sci, Inst Measurement Sci, Bratislava, Slovakia

[21] Oklahoma State Univ, Dept Psychol, Stillwater, OK 74078 USA

[22] St Thomas Univ, Fac Stat, Bogota, Colombia

[23] Univ Connecticut, Coll Hlth Agr & Nat Resources, Dept Allied Hlth Sci, Storrs, CT USA

[24] Univ Simon Bolivar, Dept Biol Organismos, Caracas, Venezuela

[25] New Bulgarian Univ, Dept Cognit Sci & Psychol, Sofia, Bulgaria

[26] Natl Res Tomsk State Univ, Tomsk, Russia

[27] Univ Ottawa, Dept Biochem Microbiol & Immunol, Ottawa, ON, Canada

[28] Oikostat GmbH, Ettiswil, Switzerland

[29] Senshu Univ, Sch Human Sci, Kawasaki, Kanagawa, Japan

[30] Leibniz Inst Wissensmedien, Multimodal Interact Lab, Tubingen, Germany

[31] Consejo Nacl Invest Cient & Tecn, Mar Del Plata, Buenos Aires, Argentina

[32] Univ Nacl Mar Del Plata, Fac Psicol, Mar Del Plata, Buenos Aires, Argentina

[33] Pontificia Univ Catolica Valparaiso, Valparaiso, Chile

[34] Univ Tecnol Chile INACAP, Vicerrectoria Invest & Desarrollo, Santiago, Chile

[35] Magna Graecia Univ Catanzaro, Dept Med & Surg Sci, Catanzaro, Italy

[36] Univ Campania Luigi Vanvitelli, Econ Dept, Capua, Italy

[37] Univ Hosp Basel, Dept Psychosomat Med, Basel, Switzerland

[38] Univ Basel, Basel, Switzerland

[39] Int Psychoanalyt Univ, Div Clin Psychol & Cognit Behav Therapy, Berlin, Germany

[40] Univ Basel, Div Clin Psychol & Epidemiol, Dept Psychol, Basel, Switzerland

[41] Univ Grenoble Alpes, CNRS, LPNC, Grenoble, France

[42] Univ Ghent, Dept Expt Clin & Hlth Psychol, Ghent, Belgium

[43] New Mexico State Univ, Dept Math Sci, Las Cruces, NM 88003 USA

[44] Univ Fed Pernambuco, Dept Stat, Computat Stat Lab CAST, Recife, PE, Brazil

[45] Massey Univ, Business Sch, Albany, New Zealand

[46] Univ Wurzburg, Dept Psychol 3, Wurzburg, Germany

[47] Univ Nacl Autonoma Mexico, Sch Psychol, Mexico City, DF, Mexico

[48] Univ Porto, Fac Engn, CONSTRUCT LESE, Porto, Portugal

[49] FOM Hsch Oekon & Management, Essen, Germany

[50] Univ Tubingen, LEAD Grad Sch, Tubingen, Germany

来源：

FRONTIERS IN PSYCHOLOGY | 2018年 / 9卷

基金：

瑞士国家科学基金会;

关键词：

statistical significance; null hypothesis testing; p-value; significance testing; decision making; P-VALUES; STATISTICAL SIGNIFICANCE; CONFIDENCE-INTERVALS; SCIENCE; POWER; REPLICATION; PSYCHOLOGY; GUIDE;

D O I：

10.3389/fpsyg.2018.00699

中图分类号：

B84 [心理学];

学科分类号：

04 ; 0402 ;

摘要：

We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new discoveries and the progress of science. Given that blanket and variable alpha levels both are problematic, it is sensible to dispense with significance testing altogether. There are alternatives that address study design and sample size much more directly than significance testing does: but none of the statistical tools should be taken as the new magic method giving clear-cut mechanical answers. Inference should not be based on single studies at all, but on cumulative evidence from multiple independent studies. When evaluating the strength of the evidence, we should consider, for example, auxiliary assumptions, the strength of the experimental design, and implications for applications. To boil all this down to a binary decision based on a p-value threshold of 0.05, 0.01, 0.005, or anything else, is not acceptable.

引用

页数：7

共 64 条

[1] Estimating the reproducibility of psychological science [J].

Aarts, Alexander A. ;

Anderson, Joanna E. ;

Anderson, Christopher J. ;

Attridge, Peter R. ;

Attwood, Angela ;

Axt, Jordan ;

Babel, Molly ;

Bahnik, Stepan ;

Baranski, Erica ;

Barnett-Cowan, Michael ;

Bartmess, Elizabeth ;

Beer, Jennifer ;

Bell, Raoul ;

Bentley, Heather ;

Beyan, Leah ;

Binion, Grace ;

Borsboom, Denny ;

Bosch, Annick ;

Bosco, Frank A. ;

Bowman, Sara D. ;

Brandt, Mark J. ;

Braswell, Erin ;

Brohmer, Hilmar ;

Brown, Benjamin T. ;

Brown, Kristina ;

Bruening, Jovita ;

Calhoun-Sauls, Ann ;

Callahan, Shannon P. ;

Chagnon, Elizabeth ;

Chandler, Jesse ;

Chartier, Christopher R. ;

Cheung, Felix ;

Christopherson, Cody D. ;

Cillessen, Linda ;

Clay, Russ ;

Cleary, Hayley ;

Cloud, Mark D. ;

Cohn, Michael ;

Cohoon, Johanna ;

Columbus, Simon ;

Cordes, Andreas ;

Costantini, Giulio ;

Alvarez, Leslie D. Cramblet ;

Cremata, Ed ;

Crusius, Jan ;

DeCoster, Jamie ;

DeGaetano, Michelle A. ;

Della Penna, Nicolas ;

den Bezemer, Bobby ;

Deserno, Marie K. .

SCIENCE, 2015, 349 (6251)

[2]

Amrhein V., 2018, PEERJ PREPRINTS, V6, DOI [10.7287/peerj.preprints.26857v1, DOI 10.7287/PEERJ.PREPRINTS.26857V1]

[3] Remove, rather than redefine, statistical significance [J].

Amrhein, Valentin ;

Greenland, Sander .

NATURE HUMAN BEHAVIOUR, 2018, 2 (01) :4-4

[4] The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research [J].

Amrhein, Valentin ;

Korner-Nievergelt, Franzi ;

Roth, Tobias .

PEERJ, 2017, 5

[5]

[Anonymous], 1979, ROBUSTNESS STAT

[6]

Balluerka N., 2005, METHODOLOGY-EUR, V1, P55, DOI [10.1027/1614-1881.1.2.55, DOI 10.1027/1614-1881.1.2.55]

[7] Redefine statistical significance [J].

Benjamin, Daniel J. ;

Berger, James O. ;

Johannesson, Magnus ;

Nosek, Brian A. ;

Wagenmakers, E. -J. ;

Berk, Richard ;

Bollen, Kenneth A. ;

Brembs, Bjoern ;

Brown, Lawrence ;

Camerer, Colin ;

Cesarini, David ;

Chambers, Christopher D. ;

Clyde, Merlise ;

Cook, Thomas D. ;

De Boeck, Paul ;

Dienes, Zoltan ;

Dreber, Anna ;

Easwaran, Kenny ;

Efferson, Charles ;

Fehr, Ernst ;

Fidler, Fiona ;

Field, Andy P. ;

Forster, Malcolm ;

George, Edward I. ;

Gonzalez, Richard ;

Goodman, Steven ;

Green, Edwin ;

Green, Donald P. ;

Greenwald, Anthony ;

Hadfield, Jarrod D. ;

Hedges, Larry V. ;

Held, Leonhard ;

Ho, Teck Hua ;

Hoijtink, Herbert ;

Hruschka, Daniel J. ;

Imai, Kosuke ;

Imbens, Guido ;

Ioannidis, John P. A. ;

Jeon, Minjeong ;

Jones, James Holland ;

Kirchler, Michael ;

Laibson, David ;

List, John ;

Little, Roderick ;

Lupia, Arthur ;

Machery, Edouard ;

Maxwell, Scott E. ;

McCarthy, Michael ;

Moore, Don ;

Morgan, Stephen L. .

NATURE HUMAN BEHAVIOUR, 2018, 2 (01) :6-10

[8]

Berk R.A., 2003, LAW PUNISHMENT SOCIA, V2nd, P235

[9] Statistical significance and clinical relevance - The importance of power in clinical trials in dermatology [J].

Bhardwaj, SS ;

Camacho, F ;

Derrow, A ;

Fleischer, AB ;

Feldmann, SR .

ARCHIVES OF DERMATOLOGY, 2004, 140 (12) :1520-1523

[10] Significance Testing Needs a Taxonomy: Or How the Fisher, Neyman-Pearson Controversy Resulted in the Inferential Tail Wagging the Measurement Dog [J].

Bradley, Michael T. ;

Brand, Andrew .

PSYCHOLOGICAL REPORTS, 2016, 119 (02) :487-504

← 1 2 3 4 5 6 7 →