The dubious benefits of multi-level modeling

This paper presents an argument against the wider adoption of complex forms of data analysis, using multi- level modeling (MLM) as an extended case study. MLM was devised to overcome some deficiencies in existing datasets, such as the bias caused by clustering. The paper suggests that MLM has an unclear theoretical and empirical basis, has not led to important practical research results, is largely unnecessary due to the availability of a range of alternatives, and is therefore overly complex, making research reports harder to read and understand for a general audience for no apparent analytic gain. Above all, the paper shows via examples that MLM has made so little difference in practice that it is worth us also considering its analytic costs. These costs include the promotion of an educational form of `asterix economics', the creation of unworkable premises such as denying the existence of population data, and the tension between the contradictory need for both a large and small number of cases at each sub-level. The paper concludes by outlining a substantial number of alternative methods of analysis which can have the same effect as MLM in examining structures in the data or overcoming the problems caused by clustering. Many of these alternatives are easier to use and understand, do not require specialist software, and avoid the problems-such as having to ignore cases with missing variables-created by the use of MLM.