The effective number of parameters in post hoc models

Although a parsimonious post hoc model may involve relatively few parameters, many other potential model parameters may, in effect, be fixed at 0 after examining the data. Because these fixed parameters could have been varied if the data had come out differently, one could argue that they should be included when assessing the complexity of the model. On the other hand, intuitively, it seems that some advantage should accrue to identifying a simple and compelling description of the data, even if it is post hoc. This problem was considered here in the context of factorial designs in which a potentially parsimonious description of the results consists of a limited set of simple effects. Monte Carlo simulations were used to establish the effective numbers of parameters for various classes of such simple, but post hoc, models.