New estimation and feature selection methods in mixture-of-experts models

We study estimation and feature selection problems in mixture of-experts models An l(2) penalized maximum likelihood estimator is proposed as an alternative to the ordinary maximum likelihood estimator The estimator is particularly advantageous when fitting a mixture-of experts model to data with many correlated features It is shown that the proposed estimator is root n consistent and simulations show its superior finite sample behaviour compared to that of the maximum likelihood estimator For feature selection two extra penalty functions are applied to the l(2)-penalized log likelihood function The proposed feature selection method is computationally much more efficient than the popular all-subset selection methods Theoretically it is shown that the method is consistent in feature selection and simulations support our theoretical results A real-data example is presented to demonstrate the method The Canadian Journal of Statistics 38 519-539 2010 (C) 2010 Statistical Society of Canada