Agreement Between Pharmacists for Problem Identification: An Initial Quality Measurement of Cognitive Services

BACKGROUND: Recent changes in national reimbursement policies expand the ability of pharmacists to seek reimbursement for cognitive services. The quality of. pharmacist-provided cognitive services has, until now, remained unassessed. Pharmacists should demonstrate the quality and value of their work to ensure the continued and expanded acceptance of reimbursement for their services. A preliminary step in assessing quality is to compare agreement between pharmacists for basic problem identification. OBJECTIVE: To quantify agreement between pharmacist reviewers for problem identification among Utah Medicaid recipients. METHODS: Five pharmacists retrospectively reviewed drug regimens, patient characteristics, diagnosis codes, and procedures for 80 Medicaid patients in September 2008 and identified drug-related problems (DRPs) in 15 predetermined categories. Data for each patient were reviewed twice, and each combination of 2 pharmacists reviewed the same 8 patients' information. We calculated a reliability coefficient to compare the number of DRPs identified and used prevalence and bias adjusted kappa (PABAK) to determine interrater reliability for the presence of a specific DRP. RESULTS: Of the 15 DRPs categorized by pharmacist reviewers, 1 (untreated indications) had a PABAK coefficient of 0.20, indicating a relatively low level of agreement between reviewers. All other DRP categories had good to excellent agreement, with PABAK coefficients ranging between 0.43 and 0.98. CONCLUSIONS: Pharmacist reviewers exhibited less variability in DRP identification or categorization than had been expected for most categories. This work supports the conclusion that pharmacists in our center provide a basic and necessary level of quality for problem assessment. Future work is needed to document the impact of this quality on patient outcomes.