Measuring Skill Growth and Evaluating Change: Unconditional and Conditional Approaches to Latent Growth Cognitive Diagnostic Models

Abstract

During the past decade, cognitive diagnostic models (CDMs) have become prevalent in providing diagnostic information for learning. Cognitive diagnostic models have generally focused on single cross-sectional time points. However, longitudinal assessments have been commonly used in education to assess students' learning progress as well as evaluating intervention effects. Thus, it becomes natural to identify longitudinal growth in skills profiles mastery, which can yield meaningful inferences on learning. This study proposes longitudinal CDMs that incorporate latent growth curve modeling and covariate extensions, with the aim to measure the growth of skills mastery and to evaluate attribute-level intervention effects over time. Using real-world data, this study demonstrates applications of unconditional and conditional latent growth CDMs. Simulation studies show stable parameter recovery and classification of latent classes for different sample sizes. These findings suggest that building on the well-established growth modeling frameworks, applications of covariate-based longitudinal CDM can help understand the effect of explanatory factors and intervention on the change of attribute mastery.

Longitudinal Reliability of Milestones-Based Learning Trajectories in Family Medicine Residents

“… 15 , 16 , 17 , 18 Learning trajectories represent longitudinal patterns of developmental progress, measuring growth and acquisition of competencies over time. 15 , 16 In this regard, identifying meaningful patterns that reveal inflection points in learner’s developmental progress and examining factors influencing the variability of milestones level (eg, program-level effects) can be informative. Although different patterns of developmental progress may exist, a learner may improve consistently during the initial phase of training but plateau at later stages; learners may also progress with varying inflection points (shift in the direction of the slope) during training when their performance stagnates or decreases.…”

“…[9][10][11][12][13][14] Longitudinal assessments can be described as learning trajectories to measure the developmental progression of learners. [15][16][17][18] Learning trajectories represent longitudinal patterns of developmental progress, measuring growth and acquisition of competencies over time. 15,16 In this regard, identifying meaningful patterns that reveal inflection points in learner's developmental progress and examining factors influencing the variability of milestones level (eg, program-level effects) can be informative.…”

A Longitudinal Diagnostic Model with Hierarchical Learning Trajectories

“…To provide theoretical support for the concept of a longitudinal learning diagnosis, several longitudinal learning diagnosis models (LDMs) have been proposed in recent years (for a review, see Pan, Qin, & Kingston, 2020; Zhan, 2020b). Existing longitudinal LDMs can be divided into two main types: latent transition analysis‐based models (e.g., Kaya & Leite, 2017; Li, Cohen, Bottge, & Templin, 2016; Madison & Bradshaw, 2018; Wang, Yang, Culpepper, & Douglas, 2018; Wen, Liu, & Zhao, 2020; Zhang & Wang, 2019) and higher‐order latent structure‐based models (e.g., Lee, 2017; Lin, Xing, & Park, 2020; Huang, 2017; Pan et al., 2020; Zhan, 2020c; Zhan, Jiao, Liao et al., 2019). The former estimates the transition probabilities from one latent attribute (profile) to another or the same latent attribute (profile).…”

Variational Bayes Inference for Hidden Markov Diagnostic Classification Models

“…Zhan, Jiao, Liao, et al (2019) employed a similar strategy in the development of a higher-order longitudinal variant of the DINA model (Junker & Sijtsma, 2001;Macready & Dayton, 1977;Maris, 1999), and Zhan, Jiao, Man, et al (2019) demonstrated how to estimate the longitudinal DINA model under a Bayesian setting. Lin et al (2020) introduced a growth curve approach for modeling higher order traits where covariates are introduced into the continuous component. Most recently, Zhan, Ma, et al (2020) developed a higher-order latent trait DCM for hierarchical attribute structures, Zhan (2020a) introduced a Markov estimation strategy for immediate diagnostic feedback, and Zhan (2021) proposed a longitudinal probabilistic DCM which treats attributes as probabilistic entities.…”