Development of a validated exam to assess physician transfusion medicine knowledge

Background There is evidence that physicians lack adequate transfusion medicine knowledge. To design needs-based educational interventions to address this gap, a validated assessment tool is required. Previously published exams have not been created or validated using rigorous psychometric methods. Study Design and Methods A modified Delphi method was used to achieve consensus regarding the essential knowledge and skills for physicians who transfuse blood products. To ensure content validity, members of an international organization of transfusion medicine experts (Biomedical Excellence for Safer Transfusion [BEST] Collaborative) participated in the exam design process. An exam, based on the most highly rated topics, was created and administered to individuals with a priori expected basic, intermediate, and expert levels of transfusion medicine knowledge. Rasch analysis, a psychometric technique used in high-stakes medical licensure and board testing, was used to determine exam accuracy and precision. Results Thirty-six topics achieved ratings sufficient to be considered for inclusion in the exam (content validity index > 0.8). A 23-question exam was administered to 49 individuals. Mean scores for individuals with expected basic, intermediate, and expert knowledge were 42, 62, and 82%, respectively (p < 0.0001). The exam achieved good fit with the Rasch model. Conclusion A validated exam has now been created to accurately assess transfusion medicine knowledge. This exam can be used to determine knowledge deficits and assist in the design of curricula to improve blood product utilization.