Employing the Payback Framework to Assess Implementation Science Research Utilization: Lessons From the USAID's PEPFAR HIV/AIDS Implementation Science Awards

Background: Stakeholders question whether implementation science (IS) is successful in conducting rigorous science that expedites the inclusion of health innovations into policies and accelerates the pace and scale of health service delivery into clinical and public health practice. Using the Payback Framework (PF) for research utilization (RU), we assessed the impact of USAID's IS investment on a subset of studies examining HIV prevention, care, and treatment. Setting: Selected USAID-funded IS awards implemented between 2012 and 2017 in 9 sub-Saharan African countries. Methods: A modified version of a RU framework, the PF, was applied to 10 USAID-funded IS awards. A semistructured, self-administered/interviewer-administered questionnaire representing operational items for the 5 categories of the modified PF was used to describe the type and to quantify the level of payback achieved. The raw score was tallied within and across the 5 PF categories, and the percentage of "payback" achieved by category was tabulated. Distribution of payback scores was summarized by tertiles. Results: Knowledge production had the highest level of payback (75%), followed by benefits to future research (70%), benefits to policy (45%), benefits to health and the health system (18%), and broader economic benefits (5%). Conclusions: All awards achieved some level of knowledge production and benefits to future research, but translation to policy and programs was low and variable. We propose the use of policy, health system, and economic monitoring indicators of RU throughout the research process to increase IS studies' impact on health practice, programs, and policy.