Evaluation of Peer Review of Percutaneous Coronary Intervention Operator Performance

BACKGROUND: Case-based peer review of percutaneous coronary intervention (PCI) is used by many hospitals for quality improvement and to make decisions regarding physician competency. However, there are no studies testing the reliability or validity of peer review for PCI performance evaluation. METHODS: We recruited interventional cardiologists from 12 Veterans Affairs Health System facilities throughout the United States to provide PCI cases for review. Ten reviewers performed blinded reviews such that each case was reviewed twice. Cases were rated on a scale of 1 to 5 (with 5 being the best) for 6 care domains (Appropriateness, Lesion Suitability, Strategy, Technical Performance, Outcome, and Documentation) with a summary performance score calculated as the average of all domains. Separately, reviewers determined whether the standard of care was met. Interobserver reliability of the summary performance score was calculated using interclass correlation coefficient. We examined procedural complications and 30-day mortality and major adverse cardiac events for all PCIs performed by these operators from 2019 to 2022 when stratified in tertiles by summary performance score. RESULTS: Of the 65 cases provided by 13 operators, the mean summary performance score was 3.90 (SD=0.78) out of 5. The interclass correlation coefficient was 0.53, indicating moderate interobserver reliability. For 19 cases (29.2%), 1 reviewer indicated that the performance did not meet the standard of care; however, the second reviewer disagreed in all these cases. Average performance scores ranged from 3.35 to 4.38. Among the 3390 PCIs performed by reviewed cardiologists from 2019 to 2022, the lowest-rated tertile had higher rates of complications (2.9% versus 1.8%, P<0.01) and major adverse cardiac events (10.6% versus 8.0%, P<0.01) compared with the highest-rated tertile. CONCLUSIONS: Case-based peer review identifies variation in physician performance that is correlated with PCI outcomes. However, reviewer disagreements about the standard of care raise concerns about the use of peer review for high-stakes assessments of physician competency.