On the usefulness of CDA-based score reporting: implications for self-regulated learning

Abstract

In cognitive diagnostic modeling research, one area that has not had enough research interests is remedial learning or instruction based on the information provided by cognitive diagnostic assessment (CDA). The present study tries to address this research gap by looking into the usefulness of the fine-grained score reports based on CDA in two different ways, i.e., a post-test and a survey inquiring about the perceived effectiveness of the score report that provided the skill profile of individual students. Another significance of the current research is that it attempted to introduce cognitive diagnostic assessment into a regular school exam unlike most previous studies that retrofitted to the existing tests. College students in Korea participated in the study, who were encouraged to do self-regulated learning utilizing the detailed information in the CDA-based score report. The results of the post-test and the survey were positive overall, supporting the utility of CDA-generated performance reports. The article ends with some suggestions for future research based on the limitations of the study.

Improving Instructional Decision‐Making Using Diagnostic Classification Models

“…However, the score reports provided for DCM-based assessments can also benefit from providing additional information beyond mastery data. Some reports include information about item responses in addition to mastery classifications (e.g., Maas et al, 2022;Rupp et al, 2010;Yi, 2021) or draw group comparisons (e.g., Du & Ma, 2021). Others aggregate the test taker's mastery information for conceptually related attributes or to provide an overall performance level in a subject (e.g., Du & Ma, 2021;Thompson et al, 2019).…”

“…Parents describe using mastery results to support student learning at home (Gane & Clark, 2023). University students describe using the results to self-regulate their own learning and to prepare for exams (Yi, 2021). When mastery information is aggregated to provide reports at the school, district, or state level, the data can inform programmatic decision-making and resource allocation determinations (DLM Consortium, 2022).…”