Relative Performance of Frequentist and Bayesian Methods for Incorporating External Controls: A Case Study with Patient Level Data from the DapaHF Trial

Some populations, such as rare diseases, cannot be studied in randomized clinical trials due to feasibility or ethical considerations. One way to address this is to incorporate external data to augment what is learned in the trial about a standard of care control arm. Statistically, there are two broad families of approaches for incorporating external controls for comparisons, propensity score-based methods (PSM) and Bayesian dynamic borrowing (BDB) methods. We evaluate these methods in terms of bias and precision with patient-level data from a large cardiovascular trial. We consider a hybrid trial setting, where external data augments a concurrently randomized control arm, and a single-arm trial using external data as a formal comparator. We evaluate performance with and without systemic biases between the trial and the external controls. The performance of PSM depends on the extent to which covariates are associated with the outcome. The performance of BDB depends on the choice of modeling parameters. Overall, there is a precision-bias tradeoff in the use of external data. In practice, it is appropriate that the observed treatment effect needs to not just achieve statistical significance, but also qualitatively overwhelm the possibility that the observed treatment effect is driven by systematic differences between data sources.