Understanding method variance in Multitrait-Multirater performance appraisal matrices: Examples using general impressions and interpersonal affect as measured method factors

As multirater performance assessment becomes more common, it becomes more important to understand the unique perspectives (and unique rating variance) of different raters. In this article, an approach for investigating the causes (e.g., general impressions and interpersonal affect) of rater-specific method variance in Multitrait-Multirater (MTMR) performance appraisal matrices is described and demonstrated. The approach is a combination of confirmatory factor analysis and the use of measured method factors. Results of demonstrations with 2 previously published MTMR datasets (Holzbach, 1978; Tucker, Cline, & Schmitt, 1967) suggest that both general impressions and interpersonal affect account for significant method variance.