Converging on quality: Examining multiple measures of teaching effectiveness

Abstract

The present study explores the convergent and predictive validity for several widely used measures of teaching quality from the Measures of Effective Teaching Project (Bill and Melinda Gates Foundation, 2009-2011). Specifically, the Classroom Assessment Scoring System (CLASS; Pianta, Hamre, & Mintz, 2012), the Framework for Teaching (FFT; Danielson Group, 2013), and the Tripod Student Perceptions Scale (Tripod; Ferguson, 2008) were examined. Correlations among measures were assessed by developmental level and content area (elementary mathematics N = 70; elementary English language arts N = 101; middle school mathematics N = 291, middle school English language arts N = 280). Both average scores and score variability (i.e., coefficient of variation) for the CLASS, FFT, and Tripod were used to predict value-added models (VAM), a high-stakes measure of students' academic growth. For elementary mathematics and ELA, findings indicated the CLASS and FFT exhibited moderate convergent validity while divergent validity was found between the Tripod and the CLASS and FFT. Across content areas in middle school grades, the CLASS, FFT, and Tripod exhibited moderate to high-moderate convergent validity. Average student and observer scores were positively related to VAM scores, whereas variability in scores demonstrated negative relations to VAM scores. Implications of findings for teacher evaluation and professional development are discussed. For decades, practitioners, researchers, and policy makers have endeavored to generate measures that capture "effective teaching" (Stronge, Ward, & Grant, 2011). Teachers have the potential to play a pivotal role in the academic and social-emotional development of their students (Pianta, 1999), yet education research indicates there is considerable variation in the quality of instruction students receive within and across classrooms (Chetty, Friedman, & Rockoff, 2011; Cohen & Goldhaber, 2016; Cohen, Ruzek, & Sandilos, 2018). In the United States, evaluation of effective teaching has propelled forward with the adoption of federal policies that provide incentives based on teacher qualifications and student achievement, such as the Teacher Incentive Fund (Heyburn, Lewis, & Ritter, 2010) and Race to the Top (U.S. Department of Education, 2009). More recently, the Every Student Succeeds Act (ESSA, 2015) stipulated that teacher evaluation systems should include multiple measures of teacher effectiveness that inform instructional planning and professional growth opportunities, underscoring the value of accumulating convergent and predictive psychometric evidence for various measures of instructional quality. Despite increased national attention, consensus about how to best measure teacher effectiveness has yet to be established

Evaluation of Teacher Practices and Student Achievement in High-Poverty Schools

“…The teacher CSAS scores were averaged across observations as part of summative evaluation procedures and used as predictors of student mathematics and reading achievement scores. Research has supported averaging scores of classroom observation tools to assess the relationship of classroom practices to student outcomes (e.g., Sandilos et al, 2019).…”

The Role of Language Teacher Metacognition and Executive Function in Exemplary Classroom Practice

“…Though there is no real consensus on how to best capture such notions (e.g., through observations, student reports), there is some agreement on which practices are likely to be high leverage and thus more impactful across classroom contexts featuring students with varying needs. Reviews of this body of research suggest that effective teaching can be described, in part, “through multidimensional assessments of teachers’ warmth, rigor, classroom management, and provision of instructional supports” (Sandilos et al., 2019, p. 11). This is also reflected in recent work that adapts such frameworks for thinking about the degree to which teachers’ classroom practice is engaging, demanding, and supportive of students’ development in our own field (e.g., Coady et al., 2020).…”

The Increased Learning Effectiveness Through Strengthening Organizational Climate and Pedagogical Competency

“…The effectiveness of learning is seen as the success of teacher to facilitate growth in an academic content area [7]. In relation to facilitating the growth of students, the effectiveness of learning is seen as the level or the degree where a problem or process succeeds in producing pleasant and expected results [8] by determining actions, choosing the right goals and methods for achievement the aim.…”

“…Table 4 shows several indicators that need to be improved, (1) job challenges, (2) autonomy, (3) student characteristics, (4) evaluation of results, (5) curriculum development, (6) preparation, (7) Correlation coefficient between organizational climate and learning effectiveness, including the strength of the relationship that is high (ry1=0.841) with a significance level of 29.38 (very significant), whose downloading indicate there is a significant positive relationship between the variables of organizational climate and learning effectiveness. Organizational climate describes the state of an organization.…”