Model-Based Incremental Validity

As an important facet of construct validity, incremental validity has been the focus of many applied investigations across a wide array of disciplines. Unfortunately, traditional methodological approaches for studying incremental validity, typically rooted in multiple regression, have many limitations that can hinder such assessments. In the current work, a strategy based in structural equation modeling is offered that greatly expands researchers' ability to investigate incremental validity of multiple individual predictors or blocks of predictors all within a single structural model. Models for four different research scenarios are presented, where the predictors of focal interest are: (a) individual measured predictors, (b) individual latent predictors, (c) blocks of measured predictors, and (d) blocks of latent predictors. Technical details of model specifications and model constraints are provided, and flexible extensions to other interesting questions (e.g., comparisons across populations) are discussed. Two empirical examples are included to illustrate the application of the proposed methods in different applied settings; complete Mplus and R syntax for both illustrative examples is supplied. Translational Abstract Researchers often care about whether variables or sets of variables contribute to predicting an outcome above and beyond other variables, thereby seeking to evaluate their incremental validity. Unfortunately, traditional regression-based methods have a number of limitations that restrict the amount and quality of information that can be gleaned in such an assessment. In the current work, a strategy based in structural equation modeling is offered that greatly enhances researchers' ability to investigate incremental validity, where the predictors of focal interest can be measured variables or latent constructs, and where they can be studied as individual predictors or in blocks. With the proposed analytical approach, researchers need only a single structural model to assess the incremental validity of multiple predictors predicting one or more outcomes. The basic structural model can also be flexibly extended to answer other interesting questions concerning incremental validity, such as differences in incremental validity patterns across multiple populations. Two empirical examples are included in this article to demonstrate how to apply the proposed methods in different applied research scenarios. Complete Mplus and R syntax is supplied for both illustrative examples, as templates for readers to adapt in their own studies.