Meta-analytic methods to test relative efficacy

In the context of multiple treatments for a particular problem or disorder, it is important theoretically and clinically to investigate whether any one treatment is more effective than another. Typically researchers report the results of the comparison of two treatments, and the meta-analytic problem is to synthesize the various comparisons of two treatments to test the omnibus null hypothesis that the true differences of all particular pairs of treatments are zero versus the alternative that there is at least one true nonzero difference. Two tests, one proposed by Wampold et al. (Psychol. Bull. 122:203–215, 1997) based on the homogeneity of effects, and one proposed here based on the distribution of the absolute value of the effects, were investigated. Based on a Monte Carlo simulation, both tests adequately maintained nominal error rates, and both demonstrated adequate power, although the Wampold test was slightly more powerful for non-uniform alternatives. The error rates and power were essentially unchanged in the presence of random effects. The tests were illustrated with a reanalysis of two published meta-analyses (psychotherapy and antidepressants). It is concluded that both tests are viable for testing the omnibus null hypothesis of no treatment differences.