How do young students with different profiles of reading skill mastery, perceived ability, and goal orientation respond to holistic diagnostic feedback?

Abstract

One critical issue with cognitive diagnostic assessment (CDA) lies in its lack of research evidence that shows how diagnostic feedback from CDA is interpreted and used by young students. This mixed methods research examined how holistic diagnostic feedback (HDF) is processed by young learners with different profiles of reading skills, goal orientations, and perceived ability. HDF provides three learner profiles: learners' current skill mastery levels; self-assessed skill proficiency; and goal orientations. It also has a section for plans for future learning. A total of 44 Grades 5 and 6 students (aged 11-12) from two classrooms, their parents and teacher received individually customized HDF reports. Students' reading skill mastery profiles were determined based on the application of cognitive diagnostic modeling to their performance on a provincial reading achievement measure, while their perceived ability and goal orientation profiles were created by using self-assessment and goal-orientation questionnaires. Students and parents provided written responses to their HDF reports.The study findings show the dynamic influence of young students' profiles on the ways in which they perceive, interpret and use HDF. Students' responses to diagnostic feedback did not differ substantially across reading mastery levels; however, psychological factors most strongly impacted the efficacy of learner feedback processing. Furthermore, the result that it was not students' actual goal orientations but their perceived parent goal orientations that showed significant relationships with their skill mastery levels strongly indicates that young students' responses to HDF are likely to be influenced by broader learning environments, and such influences are further filtered through their own perceptions. Understanding students' interactions with diagnostic feedback is critical for maximizing its effect because their perceptions about ability and orientations to learning strongly influence the ways in which they process diagnostic feedback on their learning.

The role of goal orientations in students’ perceptions of classroom assessment in higher education

“…A major observation that requires attention in these findings is that a number of students in the performance oriented group were in-service teachers (as compared to the mastery group that had pre-service teachers in the majority), with significant work experience, and their classroom comprised students who were also their professional colleagues. This is understood as a potential explanation for these students to avoid public discussion or open feedback on their assessment performance, because students with performance orientation fear evaluation and tend to avoid negative judgments (Jang et al 2015). Moreover, they also believe that their abilities are fixed and uncontrollable; therefore, feedback would not make much difference to their performance (Dweck and Leggett 1988).…”

Sample Size Requirements for Applying Diagnostic Classification Models

“…Sessoms and Henson ( 2018 ) have shown that 61% of the studies of DCMs have used sample sizes >1,000 and 31% have used sample sizes of 1,000–2,000. Some research has been reported with samples as small as 44 (Jang et al, 2015 ) and 96 (Im and Yin, 2009 ). Results of these latter studies have been reported with low or negative item discrimination values.…”

On the usefulness of CDA-based score reporting: implications for self-regulated learning

“…They conclude that providing students with skills and strategies through feedback allows them to increase their self-regulation and motivation to learn. Slightly from a different angle, Jang et al (2015) looked at how holistic diagnostic feedback was interpreted by young learners with different profiles of reading skills, goal orientations, and perceived ability. What changed students' responses to diagnostic feedback were not their reading mastery levels, but psychological factors which strongly impacted the efficacy of feedback processing.…”