Variable selection in joint mean and variance models of Box-Cox transformation

In many applications, a single Box-Cox transformation cannot necessarily produce the normality, constancy of variance and linearity of systematic effects. In this paper, by establishing a heterogeneous linear regression model for the Box-Cox transformed response, we propose a hybrid strategy, in which variable selection is employed to reduce the dimension of the explanatory variables in joint mean and variance models, and Box-Cox transformation is made to remedy the response. We propose a unified procedure which can simultaneously select significant variables in the joint mean and variance models of Box-Cox transformation which provide a useful extension of the ordinary normal linear regression models. With appropriate choice of the tuning parameters, we establish the consistency of this procedure and the oracle property of the obtained estimators. Moreover, we also consider the maximum profile likelihood estimator of the Box-Cox transformation parameter. Simulation studies and a real example are used to illustrate the application of the proposed methods.