Confidence intervals and replication: Where will the next mean fall?

Confidence intervals (CIs) give information about replication, but many researchers have misconceptions about this information. One problem is that the percentage of future replication means captured by a particular Cl varies markedly, depending on where in relation to the population mean that Cl falls. The authors investigated the distribution of this percentage for sigma known and unknown, for various sample sizes, and for robust CIs. The distribution has strong negative skew: Most 95% CIs will capture around 90% or more of replication means, but some will capture a much lower proportion. On average, a 95% Cl will include just 83.4% of future replication means. The authors present figures designed to assist understanding of what CIs say about replication, and they also extend the discussion to explain how p values give information about replication.