Selecting Mathematical Tasks for Assessing Student’s Understanding: Pre-Service Teachers’ Sensitivity to and Adaptive Use of Diagnostic Task Potential in Simulated Diagnostic One-To-One Interviews

Abstract

Teachers’ diagnostic competences are regarded as highly important for classroom assessment and teacher decision making. Prior conceptualizations of diagnostic competences as judgement accuracy have been extended to include a wider understanding of what constitutes a diagnosis; novel models of teachers’ diagnostic competences explicitly include the diagnostic process as the core of diagnosing. In this context, domain-general and mathematics-specific research emphasizes the importance of tasks used to elicit student cognition. However, the role of (mathematical) tasks in diagnostic processes has not yet attracted much systematic empirical research interest. In particular, it is currently unclear whether teachers consider diagnostic task potential when selecting tasks for diagnostic interviews and how this relationship is shaped by their professional knowledge. This study focuses on pre-service mathematics teachers’ selection of tasks during one-to-one diagnostic interviews in live simulations. Each participant worked on two 30 mins interviews in the role of a teacher, diagnosing a student’s mathematical understanding of decimal fractions. The participants’ professional knowledge was measured afterward. Trained assistants played simulated students, who portrayed one of four student case profiles, each having different mathematical (mis-)conceptions of decimal fractions. For the interview, participants could select tasks from a set of 45 tasks with different diagnostic task potentials. Two aspects of task selection during the diagnostic processes were analyzed: participants’ sensitivity to the diagnostic potential, which was reflected in higher odds for selecting tasks with high potential than tasks with low potential, and the adaptive use of diagnostic task potential, which was reflected in task selection influenced by a task’s diagnostic potential in combination with previously collected information about the student’s understanding. The results show that participants vary in their sensitivity to diagnostic task potential, but not in their adaptive use. Moreover, participants’ content knowledge had a significant effect on their sensitivity. However, the effects of pedagogical content and pedagogical knowledge did not reach significance. The results highlight that pre-service teachers require further support to effectively attend to diagnostic task potential. Simulations were used for assessment purposes in this study, and they appear promising for this purpose because they allow for the creation of authentic yet controlled situations.

Cognitive and Motivational Person Characteristics as Predictors of Diagnostic Performance: Combined Effects on Pre-Service Teachers’ Diagnostic Task Selection and Accuracy

“…For PCK, we expected a relationship to diagnostic accuracy, as the interpretation of the diagnostic information gathered about the student's understanding with respect to typical student misconceptions would be primarily related to knowledge about student cognition. Based on prior findings, we did not expect a relationship between PCK and task selection (Kron et al 2021). Since the diagnostic interview had a strongly subject-related focus, we did not expect PK to have a strong influence.…”

“…Thus, selecting high-potential tasks in diagnostic processes is crucial, but teachers' diagnostic task selection has rarely been investigated. Kron et al (2021) introduce the concept of sensitivity to the diagnostic potential of tasks as a facet of diagnostic competences, defining the diagnostic potential of a task as "its ability to stimulate student responses, allowing for the generation of reliable evidence about students' mathematical understanding". This sensitivity is reflected in the tendency to prefer tasks with high diagnostic potential during diagnostic processes.…”

“…An intervention study of Ostermann et al (2018) indicates that content-specific PCK influences pre-service teachers' estimates of the difficulty of tasks in the area of functions and graphs. Investigating pre-service teachers' task selection during diagnostic interviews on decimal fractions, Kron et al (2021) report a relationship between participants' CK on decimals and their sensitivity to the diagnostic potential of tasks on decimals. Regarding the accuracy of diagnosis, research has emphasized the influence of CK as well as PCK (Binder et al 2018).…”

“…Prior findings highlight the need to foster the diagnostic competences of preservice teachers (Kron et al 2021;Praetorius et al 2012), as the acquisition of initial conceptual knowledge (so-called personal knowledge, pPCK; e.g. Carlson et al 2019), often in a declarative form during teacher education, does not automatically lead to a successful application of knowledge.…”

Integrating or Not-Integrating—That is the Question. Effects of Integrated Instruction on the Development of Pre-Service Biology Teachers’ Professional Knowledge

“…Since pre-service teachers' application of PCK did not change after the intervention, in which declarative and action-related knowledge was presented by a lecturer, more practice-oriented training forms may be more beneficial, since simply acquiring professional knowledge (either separated or integrated) might not be sufficient for enacting the knowledge to diagnose subjectspecific instruction (cf. Kron et al, 2021). Supporting the application of PCK directly in the video-based assessment tool might be one way to improve the development of integrated knowledge that is applicable to instructional situations.…”