Testing measurement reliability in older populations - Methods for informed discrimination in instrument selection and application

Objective: The authors recommend confidence intervals as measures of precision for reliability coefficients, regression modeling as supplements for such omnibus reliability statistics, and unreliability detection as a goal of reliability testing distinct from reliability inference. Methods: Illustrative reliability analyses are conducted on measures selected from a study of clinical features associated with urinary tract infection in older nursing home residents. Results: Standard methods for reliability testing (e.g., kappa coefficients) are often inappropriate for small samples, and exact methods or descriptive reliability statistics are viable alternatives. Discussion: Supplementation of omnibus statistics by loglinear regression modeling is especially appropriate for aging research because it facilitates tests of marginal homogeneity and comparisons of reliability results for relatively young and old subgroups. Latent class regression analysis is useful for older samples because multifactorial health conditions are often measured in multiple ways and assessment of their reliability can be integrated, granting certain assumptions, with validity assessment.