Using Think-Alouds for Response Process Evidence of Teacher Attentiveness

Analyzing student response processes to refine and validate a competency model and competency-based assessment task types

“…For instance, cognitive models for tasks with different difficulty levels can be generated by including appropriate hypotheses in the model, such as items with high difficulty are solved correctly by fewer students and require a longer response time than easier items (Gorin, 2006). Corresponding underlying hypotheses should also be incorporated when investigating the alignment of different task formats, such as selected-response (SR) (e.g., multiple choice questions) versus constructed-response (CR) (e.g., responses to essay questions) items (Gorin, 2006;Mo et al, 2021). If it is assumed that both response formats elicit the same response processes or address the same competencies, the cognitive models should be identical.…”

“…In this context, we would like to verify the assumptions regarding the comparability of different task variants, by examining the alignment between response processes across task types (cf., e.g., Mo et al, 2021), which are presented in the previous section:…”

Data_Sheet_1.PDF

“…In this context, it was shown that two assumed task variants with different response formats (SR vs. CR) do not address the same competence facets. This contradicts the current state of research, according to which CR tasks can be adequately represented by corresponding SR tasks through careful task design (e.g., Mo et al, 2021). It can be assumed that in our case, the instructions of the tasks differ too much, and thus, the focus is put on different competence facets.…”

“…For instance, cognitive models for tasks with different difficulty levels can be generated by including appropriate hypotheses in the model, such as items with high difficulty are solved correctly by fewer students and require a longer response time than easier items (Gorin, 2006). Corresponding underlying hypotheses should also be incorporated when investigating the alignment of different task formats, such as selected-response (SR) (e.g., multiple choice questions) versus constructed-response (CR) (e.g., responses to essay questions) items (Gorin, 2006;Mo et al, 2021). If it is assumed that both response formats elicit the same response processes or address the same competencies, the cognitive models should be identical.…”

“…In this context, we would like to verify the assumptions regarding the comparability of different task variants, by examining the alignment between response processes across task types (cf., e.g., Mo et al, 2021), which are presented in the previous section:…”