A multiple randomization testing procedure for level, trend, variability, overlap, immediacy, and consistency in single-case phase designs

We present an approach to draw multiple and powerful inferences for each data aspect of single-case ABAB phase designs: level, trend, variability, overlap, immediacy, and consistency of data patterns. We show step-by-step how effect size measures can be calculated for each data aspect and subsequently integrated as test statistics in multiple randomization tests. To control for Type I errors, we discuss three methods for adjusting the obtained p-values based on the false discovery rate: the multiple testing correction proposed by Benjamini and Hochberg (1995), the adaptive correction suggested by Benjamini and Hochberg (2000), and the correction taking into account the dependency between the tests (Benjamini & Yekutieli, 2001). We apply this approach to a published data set and compare the results to the conclusions drawn by the authors based on visual analysis. The multiple randomization testing procedure can give more detailed information about which data aspects are affected by the single-case intervention. We provide generic R-code to execute the presented analyses.