Penalized Estimation of a Finite Mixture of Linear Regression Models

Finite mixtures of linear regressions are often used in practice in order to classify a set of observations and/or explain an unobserved heterogeneity. Their application poses two major challenges. The first is about the maximum likelihood estimation, which is, in theory, impossible in case of Gaussian errors with component specific variances because the likelihood is unbounded. The second is about covariate selection. As in every regression model, there are several candidate predictors and we have to choose the best subset among them. These two problems can share a similar solution, and here lies the motivation of the present paper. The pathway is to add an appropriate penalty to the likelihood. We review possible approaches, discussing and comparing their main features.