Appropriate covariance-specification via penalties for penalized splines in mixed models for longitudinal data

A popular approach to smooth models for longitudinal data is to express the model as a mixed model, since this often leads to immediate model fitting with standard procedures. This approach is particularly appealing when truncated polynomials are used as a basis for the smoothing, as the mixed model representation is almost immediate. We show that this approach can lead to a severely biased estimate of the overall population effect and to confidence intervals with undesirable properties. We use penalization to investigate an alternative approach with either B-spline or truncated polynomial bases and show that this new approach does not suffer from the same defects. Our models are defined in terms of B-splines or truncated polynomials with appropriate penalties, but can be expressed as mixed models; this also gives access to fitting with standard procedures. We illustrate our methods with an analysis of two datasets: (a) a balanced data seton Canadian weather and (b) an unbalanced dataset on the growth of children.