Balancing Weights for Estimating Treatment Effects in Educational Studies

Inverse probability weights often are used in education to estimate causal effects in observational studies. A common practice is to estimate the probability of treatment (e.g., by logistic regression) and use the estimated treatment probabilities to estimate the weights. However, this approach can fail to lead to weights that adequately balance covariates between the treatment and control groups. Balancing weights provide one alternative method for estimating inverse probability weights that does not suffer from this limitation. Balancing weights are based on an algorithm that directly targets reducing imbalances during the estimation process, rather than improving model fit. Here, we provide an introduction to balancing weights methods. We outline the basic mechanics for the estimation of balancing weights, including the specification of a hyperparameter. We review the role of the hyperparameter and detail methods for data driven selection. We compare and contrast the analytic workflow for balancing weights with the more standard inverse probability weighting methods. We present a comparison based on a simulation study to illustrate conditions under which balancing weights perform as well as or better than weights estimated via logistic regression. We conclude with an original empirical application where we evaluate the effectiveness of Pre-K school programs in Wake County, North Carolina.