Measuring Teaching Practices at Scale: A Novel Application of Text-as-Data Methods

Abstract

Valid and reliable measurements of teaching quality facilitate school-level decision-making and policies pertaining to teachers. Using nearly 1,000 word-to-word transcriptions of fourth- and fifth-grade English language arts classes, we apply novel text-as-data methods to develop automated measures of teaching to complement classroom observations traditionally done by human raters. This approach is free of rater bias and enables the detection of three instructional factors that are well aligned with commonly used observation protocols: classroom management, interactive instruction, and teacher-centered instruction. The teacher-centered instruction factor is a consistent negative predictor of value-added scores, even after controlling for teachers’ average classroom observation scores. The interactive instruction factor predicts positive value-added scores. Our results suggest that the text-as-data approach has the potential to enhance existing classroom observation systems through collecting far more data on teaching with a lower cost, higher speed, and the detection of multifaceted classroom practices.

Influence of Teaching and the Teacher’s Feedback Perceived on the Didactic Performance of Peruvian Postgraduate Students Attending Virtual Classes During the COVID-19 Pandemic

“…The data from the confirmatory factor analysis and invariance analysis on the teaching performance questionnaire have confirmed the monitoring variable (supervision of practice) together with the variables feedback and evaluation, constituted a second-order factor called feedback, which significantly influenced student performance in relevant practice and improvement. Similarly, these confirmed dimensions match other traits from the teacher's feedback behavior reported by other authors, for example, the variables quality of feedback and using assessments in instruction (Liu and Cohen, 2021), and Supervision of student learning (Loose and Ryan, 2020).…”

“…In other words, both the explicitness of achievement criteria and the illustration (presentation of contents) are important variables for developing an instructional practice focusing on the understanding of contents (Bell et al, 2012), to promote constructive and critical interactions between them (Serbati et al, 2020), to structure didactic interactions (teacher-and/or studentcentered), and to model the skills to be developed by their students (Bazán-Ramírez et al, 2021a). Moreover, the findings of this study regarding aspects of illustration (teaching performance) match teacher performance characteristics previously reported by Liu and Cohen (2021). Analysis and problem solving, instructional dialog, engaging students in learning, modeling, strategy use, and instruction and representation of content (Liu and Cohen, 2021) imply teaching characteristics on the teacher's side.…”

“…Moreover, the findings of this study regarding aspects of illustration (teaching performance) match teacher performance characteristics previously reported by Liu and Cohen (2021). Analysis and problem solving, instructional dialog, engaging students in learning, modeling, strategy use, and instruction and representation of content (Liu and Cohen, 2021) imply teaching characteristics on the teacher's side.…”

“…The results of this study empirically support the assumption that teacher performance in the didactic interaction influences student learning (Kaplan and Owings, 2001), both in synchronous and asynchronous didactic interactions (Prince et al, 2020). The positive effect of teaching variables on achievement variables as perceived by the students has been reported by other researchers, namely, the positive and significant effect of interactive instruction (Liu and Cohen, 2021), and instructional support on the students' performance (Wang et al, 2021).…”

“…Nevertheless, in order to identify teaching, some of the authors referenced by Aiddo and Shengauan observe actions related to both teacher's behavior during class (teaching), and instruction capacity circumstances (teaching quality) (Klieme et al, 2009;Bell et al, 2012;Hafen et al, 2014;Grosse et al, 2017;Fischer et al, 2019). When referring to teaching quality, characteristics such as emotional support, instructional support, classroom organization, type of cognitive demand, and class organization and management are described, among others (Gitomer et al, 2014;Joyce et al, 2018;Liu and Cohen, 2021). On the contrary, when teaching is described as forms of teacher behavior in teaching situations, it refers to the actions deployed by the teacher during the didactic interaction to promote learning in their students (Kaplan and Owings, 2001), and has to do with the teacher's practices-for example, behavioral tendencies-in instruction and assessment during the learning processes (Bell et al, 2012;Organization for Economic Cooperation and Development, OECD, 2013;Serbati et al, 2020;Liu and Cohen, 2021).…”