Comparing Bayesian and Frequentist Approaches for Multiple Outcome Mixed Treatment Comparisons

Objectives. Bayesian statistical methods are increasingly popular as a tool for meta-analysis of clinical trial data involving both direct and indirect treatment comparisons. However, appropriate selection of prior distributions for unknown model parameters and checking of consistency assumptions required for modeling remain particularly challenging. We compared Bayesian and traditional frequentist statistical methods for mixed treatment comparisons with multiple binary outcomes. Data. We searched major electronic bibliographic databases, Food and Drug Administration reviews, trial registries, and research grant databases up to December 2011 to find randomized studies published in English that examined drugs for female urgency urinary incontinence (UI) on continence, improvement in UI, and treatment discontinuation due to harm. Methods. We describe and fit fixed and random effects models in both Bayesian and frequentist statistical frameworks. In a hierarchical model of 8 treatments, we separately analyze 1 safety and 2 efficacy outcomes. We produce Bayesian and frequentist treatment ranks and odds ratios across all drug v placebo comparisons, as well as Bayesian probabilities that each drug is best overall through a weighted scoring rule that trades off efficacy and safety. Results. In our study, Bayesian and frequentist random effects models generally suggest the same drugs as most attractive, although neither suggests any significant differences between drugs. However, the Bayesian methods more consistently identify one drug (propiverine) as best overall, produce interval estimates that are generally better at capturing all sources of uncertainty in the data, and also permit attractive rankograms that visually capture the probability that each drug assumes each possible rank. Conclusions. Bayesian methods are more flexible and their results more clinically interpretable, but they require more careful development and specialized software.