Reliability and Validity of the Psychotherapy Supervisor Development Scale: A Meta-Analytic Evaluation

The Psychotherapy Supervisor Development Scale (Watkins, Schneider, Haynes, & Nieberding, 1995) is a measure developed to assess supervisor development as outlined by the Supervisor Complexity Model (Watkins, 1990, 1993, 1994). There is preliminary evidence of adequate psychometric properties for the measure in the published literature (Barnes & Moon, 2006; Hillman, McPherson, Swank, & Watkins, 1998; Watkins et al., 1995), but it remains a scale that requires more evaluation. We conducted reliability and validity generalization meta-analyses in order to better understand the PSDS's psychometric properties. The mean internal reliability generated from 1,163 participants (6 studies) was r = 0.93, 95% CI [0.91, 0.94]. As for the scale's convergent validity, the mean effect related to supervision experience derived from 948 participants (5 studies) was g = 0.40, 95% CI [0.17, 0.62], and the mean effect related to supervision training derived from 448 participants (4 studies) was g = 1.13, 95% CI [0.53, 1.73]. For each analysis, we characterize the variance in scores across studies, and consider sample and study characteristics that are predictive of the reliability and validity coefficients. We discuss the importance of conducting sample-specific reliability analyses and make recommendations for future research on the development of supervisor competency.