Assessing moderating effect in meta-analysis: a re-analysis of top management support studies and suggestions for researchers

Meta-analysis has been increasingly used as a knowledge cumulation tool by IS researchers. In recent years many meta-analysts have conducted moderator analyses in an attempt to develop and test theories. These studies suffer from several methodological problems and, as a result, may have contributed to rather than resolved inconsistent research findings. For example, a previous meta-analysis reports that task interdependence moderates the effect of top management support to render it a non-critical component in systems implementation projects when task interdependence is low. We show that this conclusion is the result of uncorrected measurement error and an erroneous application of a fixed effects regression analysis. We discuss other pitfalls in the detection and confirmation of moderators including the use of the Q statistic and significance tests. Our recommended approach is to break the sample into subgroups and compare their credibility and confidence intervals. This approach is illustrated in a re-analysis of the top management support literature. Our results indicate that top management support is important in both high and low task interdependence groups and in fact may be equally important in both groups. Guidelines are developed to help IS researchers properly conduct moderator analyses in future meta-analytic studies. European Journal of Information Systems (2011) 20, 693-702. doi:10.1057/ejis.2011.12; published online 26 April 2011