Multiple comparisons procedures

In biomedical research, a common question posed by investigators is whether or not an outcome of interest differs significantly between multiple independent groups of subjects in the study sample. For example, in a randomized clinical trial focusing on differences in a parameter of cardiovascular health such as systolic blood pressure or heart rate that is measured on a continuum, one might make the comparison of those who received a placebo, those who received a particular active drug, and those who received a different active drug. Another example of a multiple group comparison might arise in an observational study when comparisons between categories of race or ethnicity are of interest. The statistical problem that arises from the use of multiple comparisons tests is that any subsequent tests of hypotheses will be performed on the outcome with the same data on which the global test was performed. This can result in an uncontrolled type I error rate (the rate of rejecting the null hypothesis when it should not be rejected). These tests can produce this statistical problem, which can be encountered in analyses of multiple treatment or exposure groups, multiple end points, or multiple interim analyses. This problem has been addressed from a broad perspective.1 The present report, however, will focus on the statistical analysis strategies used when the global or omnibus test of differences on a continuous outcome across the multiple groups has been performed and statistical tests contrasting subgroups are then conducted. It serves as a follow-up to an earlier article2 in the series of statistical tutorials in Circulation that addressed the use of the ANOVA in performing the global test of hypothesis for a continuous outcome. These statistical tests are often referred to as multiple comparisons procedures (MCPs). We will first present a brief review of the statistical foundations of 1-factor ANOVA and then will describe the 2 main types of MCPs with specific reference to the more commonly used MCPs. Finally, we will show a worked example of an analysis of data from a study of heart size in animals exposed to different conditions of physical exercise that will illustrate the use of 1-factor ANOVA with supplementary MCPs.. © 2008 American Heart Association, Inc.