Methodology of comparative studies on the relative effectiveness of COVID-19 vaccines: a systematic review

Objectives: This study aimed to comprehensively outline the methodological approaches used in published research comparing the vaccine effectiveness (VE) of coronavirus disease 2019 (COVID-19) vaccines. Methods: A systematic search was conducted on June 13, 2024, to identify comparative studies evaluating the effectiveness of mRNA versus non-mRNA and monovalent versus bivalent COVID-19 vaccines. We screened titles, abstracts, and full texts, collecting data on publication year, country, sample size, study population composition, study design, VE estimates, outcomes, and covariates. Studies that reported relative VE (rVE) were analyzed separately from those that did not. Results: We identified 25 articles comparing rVE between mRNA and non-mRNA COVID-19 vaccines, as well as between monovalent and bivalent formulations. Among the studies assessing VE by vaccine type, 126 did not provide rVE estimates. Comparative VE studies frequently employed retrospective cohort designs. Among the definitions of rVE used, the most common were hazard ratio and absolute VE, calculated as (1-odds ratio)x100. Studies were most frequently conducted in the United Kingdom and the United States, and the most common outcome was infection. Most targeted the general population and assessed the VE of mRNA vaccines using the AstraZeneca vaccine as a reference. A small proportion, 7.3% (n= 11), did not adjust for any variables. Only 3 studies (2.0%) adjusted for all core confounding variables recommended by the World Health Organization. Conclusion: Few comparative studies of COVID-19 vaccines have incorporated rVE methodologies. Reporting rVE and employing a consistent set of covariates can broaden our understanding of COVID-19 vaccines.