Development of an Instrument to Evaluate Residents' Confidence in Quality Improvement

Background: Practice-based learning and improvement is a core competency that all medical residents must demonstrate. Because confidence is important in translating competence into action, effective quality improvement (QI) curricula should evaluate trainees' knowledge and confidence to perform QI. Past efforts to assess educational outcomes in QI have not adequately evaluated trainees' confidence from a multidimensional perspective. Methods: Participants-732 internal medicine and family medicine residents from 42 training programs in the United States-completed the 31-item Quality Improvement Confidence Instrument (QICI), which was developed to measure confidence in six QI skill domains based on the Institute for Healthcare Improvement model of QI. Confirmatory factor analysis was performed to support construct validity. Multivariate analysis of covariance was used to examine associations between residents' QI experience and other characteristics with confidence scores. Results: Confirmatory factor analysis supported the QICI's multidimensional structure. Individual items yielded adequate variability, and reliability estimates for all six domains were high (> 0.86). On average, residents rated their confidence lowest for skills pertaining to choosing a target for improvement (specifically, using methods to evaluate interventions and to identify sources of process errors) and for testing a change in practice using specific tools for data collection and analysis. After controlling for program year and other characteristics, residents with previous QI experience reported significantly greater QI confidence. Conclusion: The QICI offers a psychometrically rigorous approach to evaluating residents' confidence levels. It can be used to gauge the appropriateness of a trainee's confidence against actual QI performance.