Observational data analysis using generalizability theory and general and mixed linear models: an empirical study of infant learning and development

Accurate evaluation of early childhood competencies is essential for favoring optimal development, as the first years of life form the foundations for later learning and development. Nonetheless, there are still certain limitations and deficiencies related to how infant learning and development are measured. With the aim of helping to overcome some of the difficulties, in this article we describe the potential and advantages of new data analysis techniques for checking the quality of data collected by the systematic observation of infants and assessing variability. Logical and executive activity of 48 children was observed in three ages (18, 21 and 24 months) using a nomothetic, follow-up and multidimensional observational design. Given the nature of the data analyzed, we provide a detailed methodological and analytical overview of generalizability theory from three perspectives linked to observational methodology: intra-and inter-observer reliability, instrument validity, and sample size estimation, with a particular focus on the participant facet. The aim was to identify the optimal number of facets and levels needed to perform a systematic observational study of very young children. We also discuss the use of other techniques such as general and mixed linear models to analyze variability of learning and development. Results show how the use of Generalizability Theory allows controlling the quality of observational data in a global structure integrating reliability, validity and generalizability.