What is the probability of replicating a statistically significant effect?

被引:53
作者
Miller, Jeff [1 ]
机构
[1] Univ Otago, Dept Psychol, Dunedin, New Zealand
关键词
CONFIDENCE-INTERVALS; PSYCHOLOGICAL-RESEARCH; GAMBLERS FALLACY; HOT HAND; P VALUES; POWER; INFERENCE;
D O I
10.3758/PBR.16.4.617
中图分类号
B841 [心理学研究方法];
学科分类号
040201 ;
摘要
If an initial experiment produces a statistically significant effect, what is the probability that this effect will be replicated in a follow-up experiment? I argue that this seemingly fundamental question can be interpreted in two very different ways and that its answer is, in practice, virtually unknowable under either interpretation. Although the data from an initial experiment can be used to estimate one type of replication probability, this estimate will rarely be precise enough to be of any use. The other type of replication probability is also unknowable, because it depends on unknown aspects of the research context. Thus, although it would be nice to know the probability of replicating a significant effect, researchers must accept the fact that they generally cannot determine this information, whichever type of replication probability they seek.
引用
收藏
页码:617 / 640
页数:24
相关论文
共 50 条
  • [21] The Fragility of Statistically Significant Results in Pediatric Orthopaedic Randomized Controlled Trials as Quantified by the Fragility Index: A Systematic Review
    Khormaee, Sariah
    Choe, Judy
    Ruzbarsky, Joseph J.
    Agarwal, Kunal N.
    Blanco, John S.
    Doyle, Shevaun M.
    Dodwell, Emily R.
    [J]. JOURNAL OF PEDIATRIC ORTHOPAEDICS, 2018, 38 (08) : E418 - E423
  • [22] Fragility of statistically significant findings from randomized clinical trials of surgical treatment of humeral shaft fractures: A systematic review
    Morris, Stephen Craig
    Gowd, Anirudh K.
    Agarwalla, Avinesh
    Phipatanakul, Wesley P.
    Amin, Nirav H.
    Liu, Joseph N.
    [J]. WORLD JOURNAL OF ORTHOPEDICS, 2022, 13 (09): : 825 - 836
  • [23] What Is In a (Non-) Significant Finding? Moving Beyond False Dichotomies
    Steenbergen, Marco R.
    [J]. SWISS POLITICAL SCIENCE REVIEW, 2019,
  • [24] Strength Is Still a Weakness in Coalition Formation: Replicating and Understanding the Strength-Is-Weakness Effect
    Wissink, Joeri
    van Beest, Ilja
    Pronk, Tila
    van de Ven, Niels
    [J]. PERSONALITY AND SOCIAL PSYCHOLOGY BULLETIN, 2022, 48 (02) : 283 - 296
  • [25] P-value and the probability of direction of effect
    Biggeri, Annibale
    Stoppa, Giorgia
    Catelan, Dolores
    [J]. EPIDEMIOLOGIA & PREVENZIONE, 2022, 46 (03): : 204 - 210
  • [26] What influences the probability of wind farm planning approval: Evidence from Ireland
    van Rensburg, Thomas M.
    Kelley, Hugh
    Jeserich, Nadine
    [J]. ECOLOGICAL ECONOMICS, 2015, 111 : 12 - 22
  • [27] Reply to the Letter to the Editor Regarding "Fragility Analysis of Statistically Significant Outcomes of Randomized Control Trials in Spine Surgery: A Systematic Review. Spine (Phila Pa 1976)"
    Muthu, Sathish
    [J]. SPINE, 2020, 45 (24) : E1707 - E1708
  • [28] Significant phonon drag effect in wide band gap GaN and AlN
    Quan, Yujie
    Chen, Yubi
    Liao, Bolin
    [J]. PHYSICAL REVIEW B, 2023, 107 (24)
  • [29] No significant effect of mortality salience on unconscious ethnic bias among the Japanese
    Otsubo, Kai
    Yamaguchi, Hiroyuki
    [J]. BMC RESEARCH NOTES, 2023, 16 (01)
  • [30] Action outcome probability influences the size of the head-fake effect in basketball
    Guldenpenning, Iris
    Jackson, Robin C.
    Weigelt, Matthias
    [J]. PSYCHOLOGY OF SPORT AND EXERCISE, 2023, 68