The accuracy of teachers' judgments for assessing young children's emerging literacy and math skills

Abstract

Because of developmental constraints on the types of assessment that can be used with young children, teachers' judgments of students' skills and abilities are a particularly important source of information in early education. The present study investigates the accuracy of these judgments by examining agreement between Prekindergarten teachers' (n = 66) judgments of children's (n = 122) emerging math and literacy skills made with teacher rating scales and children's performance on a researcher-administered demand performance assessment. Average agreement between measures was 71% for language and literacy items and 66% for math items. When disagreements occurred for language and literacy skills, they were more likely a result of teachers reporting that children had fewer skills than indicated by the demand performance assessment, rather than more. For math, teachers were equally likely to report either fewer or more math skills than indicated by the performance measure. Teachers' over-and underestimations of children's skills were related to their ratings of the children's social-emotional functioning and the type of skills being assessed. Children's gender and race were not related to teacher accuracy; neither were teacher education or years of experience.

The influence of data on teachers' judgments of students' early reading and math skills

“…However, another consistent finding with previous research (Fedora, ) was that the current sample of teachers also had a low level of accuracy when rating early reading skills, such as letter naming fluency and letter sounds fluency. Findings suggest that teachers also had a low level of accuracy when rating early math skills, which is consistent with the results of Kowalski et al (). These results need to be considered within the context of the school's instruction, what teachers are typically asking students to do within the classroom and teachers experience with the screening tools.…”

“…While teacher education and years of experience have been found to be not related to teacher accuracy (Kowalski et al, ), teachers' knowledge of and definition of the skills being assessed may impact the accuracy of their judgments. Variability and ambiguity of teacher definitions of the skills being assessed is likely to result in overestimation of their students' reading skills (Meisinger, Bradley, Schwanenfluegal, & Kuhn, ).…”

“…A meta-analysis that examined 75 studies of teacher judgments among kindergarten through twelfth grade, found that overall there were no significant differences between language arts and math (Südkamp, Kaiser, & Möller, 2012). However, recently it was found that there was a moderate agreement between pre-k teachers' judgments and an early literacy performance measure (r = 0.60) and a lower level of agreement for math skills (r = 0.46; Kowalski, Brown, Pretti-Frontczak, Uchida, & Sacks, 2018). These results are consistent with the findings of Eckert, Dunn, Codding, Begeny, and Kleinmann (2006), which found that while teachers showed low accuracy in both reading and math fluency, the correspondence between teacher ratings and curriculum-based measures was moderate to high for reading and low for math.…”

“…For example, Südkamp et al () suggested that variation in judgments may be more related to measurement error than actual variation in student performance. In addition, in previous research on teacher judgments, there is a range of sample sizes used, with some studies having only two teachers (Eckert et al, ) and others having over 60 (Kowalski et al, ), which also impacts how many students are involved in the analyses. When using correlational analyses as a method of analyzing the results, sample size is important.…”

The Association of Executive Functioning With Academic, Behavior, and Social Performance Ratings in Children With ADHD

“…The fact that this relation was particularly observed in teacher performance ratings is important because teachers have a normative group to which to compare children, and therefore a good sense of whether a child’s performance in a particular academic domain is problematic or not (Amador-Campos et al, 2006). Furthermore, teachers are able to observe student performance on a more comprehensive sample of academic content than that captured by a standardized achievement test (Kowalski et al, 2018). Therefore, their ratings can provide insight into the students’ “teachability” or ability to succeed in the regular education classroom (Kowalski et al, 2018).…”