Validation of Computerized Adaptive Testing in an Outpatient Nonacademic Setting: The VOCATIONS Trial

Objective: Computerized adaptive testing (CAT) provides an alternative to fixed-length assessments. The study validated a suite of computerized adaptive tests for mental health (CAT-MH) in a community psychiatric sample. Methods: A total of 145 adults from a community outpatient clinic, including 19 with no history of a mental disorder (control group), were prospectively evaluated with CAT for depression (CAD-MDD and CAT-DI), mania (CAT-MANIA), and anxiety symptoms (CAT-ANX). Ratings were compared with gold-standard psychiatric assessments, including the Structured Clinical Interview for DSM-IV-TR (SCID), Hamilton Rating Scale for Depression (HAM-D-25), Patient Health Questionnaire (PHQ-9), Center for Epidemiologic Studies Depression Scale (CES-D), and Global Assessment of Functioning (GAF). Results: Sensitivity and specificity for CAD-MDD were .96 and .64, respectively (. 96 and 1.00 for major depression versus the control group). CAT for depression severity (CAT-DI) correlated well with the HAM-D-25 (r = .79), PHQ-9 (r = .90), and CES-D (r = .90) and had an odds ratio (OR) of 27.88 across its range for current SCID major depressive disorder. CAT-ANX correlated with the HAM-D-25 (r =. 73), PHQ-9 (r = .78), and CES-D (r = .81) and had an OR of 11.52 across its range for current SCID generalized anxiety disorder. CAT-MANIA did not correlate well with the HAM-D-25 (r = .31), PHQ-9 (r = .37), and CES-D (r = .39), but it had an OR of 11.56 across its range for a current SCID bipolar diagnosis. Participants found the CAT-MH acceptable and easy to use, averaging 51.7 items and 9.4 minutes to complete the full battery. Conclusions: Compared with gold-standard diagnostic and assessment measures, CAT-MH provided an effective, rapidly administered assessment of psychiatric symptoms.