An empirical examination of visual analysis procedures for clinical practice evaluation

There has been a resurgence of interest in single-subject research designs and analytic tools to help clinicians detect treatment effects. The present study investigates Nugent's (2000) Visual analysis procedures, which were designed to aid practitioners in detecting clinical change for the purposes of practice evaluation. The ability of the visual procedures to detect real change in short auto-correlated data streams and the ability of the procedures to help clinicians discern cases when no actual change has occurred were evaluated. Monte Carlo analyses indicate that the power of the Visual procedures is acceptable for effect sizes of 2.25 or greater when there are at least 14 data points (7 baseline and 7 treatment) in the data set. The procedures, however, frequently lead to erroneous decisions that effects are present in data streams when, in fact, there are none. The mean type I error rate across various N's and levels of auto-correlation was .66. As they are currently designed, Nugent's visual analysis procedures make too many type I errors to be useful. (C) 2004 by The Haworth Press, Inc. All rights reserved.