Comparison of Treatment Effect Estimates From Prospective Nonrandomized Studies With Propensity Score Analysis and Randomized Controlled Trials of Surgical Procedures

Objective: We aimed to compare treatment effect estimates from NRSs with PS analysis and RCTs of surgery. Background: Evaluating a surgical procedure in randomized controlled trials (RCTs) is challenging. Nonrandomized studies (NRSs) involving use of propensity score (PS) analysis to limit bias are of increasing interest. Design: Meta-epidemiological study. Methods: We systematically searched MEDLINE via PubMed for all prospective NRSs with PS analysis evaluating a surgical procedure. Related RCTs, addressing the same clinical questions, were systematically retrieved. Our primary outcome of interest was all-cause mortality. We also selected 1 subjective outcome. We calculated the summary odds ratios (OR) for each study design, the ratio of OR (ROR) between the designs and the summary ROR across clinical questions. An ROR < 1 indicated that the experimental intervention is more favorable in NRSs with PS analysis than RCTs. Results: We retrieved 70 reports of NRSs with PS analysis and 94 related RCTs evaluating 31 clinical questions, of which 22 assessed all-cause mortality and 26 a subjective outcome. The combined ROR for all-cause mortality was 0.83 (95% confidence interval: 0.65-1.04). For subjective outcomes, the combined ROR was 1.07 (0.87-1.33). Conclusions: There was no statistically significant difference in treatment effect between NRSs with PS analysis and RCTs. Prospective NRSs with suitable and careful PS analysis can be relied upon as evidence when RCTs are not possible.