Development of self-directed activities and a validated exam for primary care advanced pharmacy practice experiences

Introduction: The impact of required self-directed activities (SDAs) measured by written exam on advanced pharmacy practice experiences (APPEs) has not been reported. The primary objective of this study was to determine the validity and reliability of a pre/post exam to measure knowledge when SDAs are incorporated into primary care APPEs. Secondary objectives included assessment of pre/post exam scores and students' perceptions of their learning through SDAs. Methods: Two faculty implemented required SDAs and a pre/post-exam in their APPEs. Face and content validity were assessed using expert consensus. A retrospective exam of student performance from May 2016 to April 2018 was performed (n = 42). Additional APPE assessment scores were also collected to determine correlations and establish criterion validity. Students who completed the APPE during May 2016 to February 2017 (n = 17) were invited to complete an anonymous, 15-item cross-sectional survey to assess student perceptions of the activities. Differences in self-rated knowledge before and after the rotation were assessed. Results: Pre-exam and post-exam scores positively correlated with journal club and the APPE grading rubric and final scores. Overall, interrater reliability was excellent. Significant improvement was seen between pre/post exam scores with a mean (SD) change of 21.5% (9.4%). Thirteen (77%) students completed the anonymous survey and rated their perceived knowledge significantly higher after completing the SDAs. Conclusions: The pre/post exam was valid, reliable, and positively correlated with other graded assignments. Significant improvements in exam scores and survey results support SDAs as an effective teaching method on primary care APPEs.