An algorithm for searching optimal variance component estimators in linear mixed models

Estimation of variance components is essential in data analysis based on a linear mixed model. Among different estimation methods, the most commonly used in practice are likelihood-based approaches: maximum likelihood and restricted maximum likelihood. These methods do not guarantee positive-unbiased estimators, however. The method of moments ensures unbiasedness but may yield negative estimates, which can cause problems when estimating fixed effects by generalized least squares. The paper develops an algorithm to derive strictly positive unbiased estimators with minimum variance in a well-defined class. The key idea is to take the method-of-moments estimator, given by a quadratic form of a symmetric matrix A, as a starting point and modify it using the class of square non-singular regularization matrices Q while preserving unbiasedness in addition to ensuring positivity. Different subclasses of structured Q are possible for convenience instead of all possible Q matrices. A search algorithm then finds a local or global optimal matrix A depending on Q and the corresponding optimal variance component estimator by minimizing the variance, a function of the unknown variance components and kurtosis parameters in the model. Availability of an optimal matrix A means that the theoretical and numerical properties of the estimator can be conveniently studied, also in small samples. The proposed method further allows finding matrices A leading to quadratic forms closely approximating the corresponding numerical values of the likelihood-based estimates. The paper investigates the dependence of variance functions on the unknown model parameters. Using two illustrative examples, Examples I and II, the paper also illustrates the use of the matrix Q and the determination of the kurtosis parameters in the search for the optimal variance component estimators by keeping the bias zero and the variance trim for the bias-variance trade-offs.(c) 2023 Elsevier B.V. All rights reserved.