Statistically significant? Let us recognize that estimates of tested effects are uncertain

被引:3
作者
Bultez, Alain [1 ]
Derbaix, Christian [1 ]
Herrmann, Jean-Luc [2 ]
机构
[1] UCLouvain, FUCaM, Louvain Sch Management, 151 Chaussee Binche, B-7000 Mons, Belgium
[2] Univ Lorraine, CEREFIGE, Nancy, France
关键词
contextualization; dichotomization; interpretation; probability; significant; threshold; uncertainty; P-VALUES; SIZE; PROBABILITY; PROPOSAL; CHOICE;
D O I
10.1177/20515707211040743
中图分类号
F [经济];
学科分类号
02 ;
摘要
Haven't all of us dreamt of concluding that our results be statistically significant, that is, characterized by a p-value lying below an arbitrary threshold, most often 5% ? In this article, we, first, deplore that p has been largely misunderstood, and that its misinterpretation has entailed a fallacious dichotomization and an understatement of the uncertainty prevailing about the effect tested. Next, we introduce and explain a brand-new - direct - measure of the plausibility of the effect under study. Then, we illustrate the relevance of this indicator by revisiting a recently published marketing research case. We also insist on the necessity to contextualize it, using complementary credibility intervals graphically contrasted. Beyond making researchers aware of the exact meaning of test-related probabilities, the delineated approach invites them to formulate their inferences with prudence and modesty acknowledging how uncertain these are.
引用
收藏
页码:82 / 105
页数:24
相关论文
共 47 条
[1]  
Agresti A., 2007, An introduction to categorical data analysis, V2
[2]   Inferential Statistics as Descriptive Statistics: There Is No Replication Crisis if We Don't Expect Replication [J].
Amrhein, Valentin ;
Trafinnow, David ;
Greenland, Sander .
AMERICAN STATISTICIAN, 2019, 73 :262-270
[3]   Retire statistical significance [J].
Amrhein, Valentin ;
Greenland, Sander ;
McShane, Blake .
NATURE, 2019, 567 (7748) :305-307
[4]  
[Anonymous], 1885, Journal of the Statistical Society of London, Jubilee Volume
[5]  
[Anonymous], 2012, Flawed science: The fraudulent research practices of social psychologist Diederik Stapel
[6]   Science is about corroborating empirical evidence, even in academic business research journals [J].
Babin, Barry J. ;
Ortinau, David J. ;
Herrmann, Jean-Luc ;
Lopez, Carmen .
JOURNAL OF BUSINESS RESEARCH, 2021, 126 :504-511
[7]   Rejection odds and rejection ratios: A proposal for statistical practice in testing hypotheses [J].
Bayarri, M. J. ;
Benjamin, Daniel J. ;
Berger, James O. ;
Sellke, Thomas M. .
JOURNAL OF MATHEMATICAL PSYCHOLOGY, 2016, 72 :90-103
[8]   Three Recommendations for Improving the Use of p-Values [J].
Benjamin, Daniel J. ;
Berger, James O. .
AMERICAN STATISTICIAN, 2019, 73 :186-191
[9]   Redefine statistical significance [J].
Benjamin, Daniel J. ;
Berger, James O. ;
Johannesson, Magnus ;
Nosek, Brian A. ;
Wagenmakers, E. -J. ;
Berk, Richard ;
Bollen, Kenneth A. ;
Brembs, Bjoern ;
Brown, Lawrence ;
Camerer, Colin ;
Cesarini, David ;
Chambers, Christopher D. ;
Clyde, Merlise ;
Cook, Thomas D. ;
De Boeck, Paul ;
Dienes, Zoltan ;
Dreber, Anna ;
Easwaran, Kenny ;
Efferson, Charles ;
Fehr, Ernst ;
Fidler, Fiona ;
Field, Andy P. ;
Forster, Malcolm ;
George, Edward I. ;
Gonzalez, Richard ;
Goodman, Steven ;
Green, Edwin ;
Green, Donald P. ;
Greenwald, Anthony ;
Hadfield, Jarrod D. ;
Hedges, Larry V. ;
Held, Leonhard ;
Ho, Teck Hua ;
Hoijtink, Herbert ;
Hruschka, Daniel J. ;
Imai, Kosuke ;
Imbens, Guido ;
Ioannidis, John P. A. ;
Jeon, Minjeong ;
Jones, James Holland ;
Kirchler, Michael ;
Laibson, David ;
List, John ;
Little, Roderick ;
Lupia, Arthur ;
Machery, Edouard ;
Maxwell, Scott E. ;
McCarthy, Michael ;
Moore, Don ;
Morgan, Stephen L. .
NATURE HUMAN BEHAVIOUR, 2018, 2 (01) :6-10
[10]   The p-Value Requires Context, Not a Threshold [J].
Betensky, Rebecca A. .
AMERICAN STATISTICIAN, 2019, 73 :115-117