Joint cognitive diagnostic modeling for probabilistic attributes incorporating item responses and response times

Compared with the conventional CDM with deterministic or binary attributes, the CDM with probabilistic attributes (probabilistic-CDM) can achieve a more refined diagnosis of attribute mastery status, which helps distinguish individual differences between students and provides more reference information for teacher feedback. However, existing probabilistic CDMs can only analyze a single modal of data-item response accuracy (RA), ignoring other modals of data such as item response times (RTs). RTs reflect the cognitive processing speed of the participant. With the popularity of computerized testing, recording RT data has become routine. However, how to use RTs in probabilistic CDM to further improve parameter estimation accuracy and enrich the diagnostic feedback information is still an unsolved methodological problem. To this end, the current study proposes three joint probabilistic CDMs based on the joint-hierarchical and joint- cross-loading cognitive diagnostic modeling approaches. First, based on joint-hierarchical modeling, the joint-hierarchical probabilistic CDM (JRT-PINC) was proposed in Study 1, which achieved the purpose of using RT to improve diagnostic accuracy. A simulation study was conducted to investigate the psychometric performance of the JRT-PINC under various simulated testing conditions, in which three independent variables, including sample size, test length, and the correlation between person parameters, were manipulated. Second, two joint-cross-loading probabilistic CDMs (CJRT-PINC-theta and CJRT-PINC-m) were proposed based on the joint-cross-loading modeling. In contrast to the JRT-PINC model, two CJRT-PINC models directly used RTs to provide information for latent abilities or attributes by introducing item-level cross-loading parameters. Two CJRT-PINC models released some conditional independence assumptions in JRT-PINC, increasing their application scope. Two simulation studies were conducted to explore their performance under different simulated conditions with different degrees of cross-loading. Third, Study 3 aims to explore the relative merits of the JRT-PINC and two CJRT-PINC models, that is, the necessity of considering cross-loading in the joint analysis of RA and RT. Finally, an empirical example was conducted to illustrate the practical applicability of the proposed models and to compare them with existing CDMs (e.g., CDMs with deterministic attributes). The simulation results mainly indicated that: (1) all three proposed models can be well recovered under different simulated conditions; (2) CJRT-PINC-theta makes fuller use of the information contained in RTs and thus improves the accuracy of the parameter estimation of the core constructs (e.g., latent ability and attributes) than CJRT-PINC-m; and (3) the adverse effects of ignoring the possible cross-loadings are more severe than redundantly considering them. The results of the empirical example indicated that: (1) probabilistic attributes provide more refined feedback on participants' mastery of attributes than deterministic attributes; and (2) two CJRT-PINC models fit this data better than the JRT-PINC model. Overall, this paper introduced RTs in probabilistic CDM for the first time and proposed three joint probabilistic CDMs based on two joint cognitive diagnostic modeling approaches. This study enriched the scope of application of probabilistic CDMS and provided methodological guidance for further refined and comprehensive diagnosis by jointly analyzing multi-modal data in technology-enhanced assessment systems.