Two-component mixtures of generalized linear mixed effects models for cluster correlated data

Finite mixtures of generalized linear mixed effect models are presented to handle situations where within-cluster correlation and heterogeneity (subpopulations) exist simultaneously. For this class of model, we consider maximum likelihood (ML) as our main approach to estimation. Owing to the complexity of the marginal loglikelihood of this model, the EM algorithm is employed to facilitate computation. The major obstacle in this procedure is to integrate over the random effects' distribution to evaluate the expectation in the E step. When assuming normally distributed random effects, we consider adaptive Gaussian quadrature to perform this integration numerically. We also discuss nonparametric ML estimation under a relaxation of the normality assumption on the random effects. Two real data sets are analysed to compare our proposed model with other existing models and illustrate our estimation methods.