Is Effective Teacher Evaluation Sustainable? Evidence from District of Columbia Public Schools

Abstract

Ten years ago, many policymakers viewed the reform of teacher evaluation as a highly promising mechanism to improve teacher effectiveness and student achievement. Recently, that enthusiasm has dimmed as the available evidence suggests the subsequent reforms had a mixed record of implementation and efficacy. Even in districts where there was evidence of efficacy, the early promise of teacher evaluation may not sustain as these systems mature and change. This study examines the evolving design of IMPACT, the teacher evaluation system in the District of Columbia Public Schools (DCPS). We describe the recent changes to IMPACT which include higher performance standards for lower-performing teachers and a reduced emphasis on value-added test scores. Descriptive evidence on the dynamics of teacher retention and performance under this redesigned system indicate that lower-performing teachers are particularly likely to either leave or improve. Corresponding causal evidence similarly indicates that imminent dismissal threats for persistently low-performing teachers increased both teacher attrition and the performance of returning teachers. These findings suggest teacher evaluation can provide a sustained mechanism for improving the quality of teaching.

Teacher Evaluation and Teacher Turnover in Equilibrium: Evidence From DC Public Schools

“…IMPACT is not mechanically responsible for most of these exits, as 63% are voluntary. However, IMPACT's identification of low-performing teachers meaningfully increases their likelihood of voluntarily exiting, as indicated by Dee et al (2019).…”

“…The DC City Council has discussed legislation that could subject IMPACT to collective bargaining (Stein, 2019a), and DCPS Chancellor Lewis Ferebee announced recently that he is conducting a review of IMPACT to explore modifications that would improve teacher evaluation in DCPS (Stein, 2019b). This article, with Dee et al (2019), shows that teaching and learning gains resulting from teacher evaluation can be sustained once these systems mature, even in the face of transitions in leadership, meaningful design modifications, implementation fatigue, competing priorities, and pressure from stakeholders. Nonetheless, about a quarter of Highly Effective teachers have concerns over the ability of IMPACT to reflect their effectiveness and identify their strengths and weaknesses, indicating room for improvement on measures of effectiveness.…”

“…Despite the changed context for teacher evaluation nationally and at DCPS, IMPACT’s incentives continue to induce low-performing teachers to exit at significantly higher rates than otherwise-similar teachers. Employing data in the years since the Dee and Wyckoff (2015) RD analysis (2012–2013 through 2015–2016), Dee et al (2019) find that Minimally Effective teachers exit at a rate that is 40% greater than otherwise-similar Developing teachers, and Developing teachers exit at a rate that is 40% greater than otherwise-similar Effective teachers. For Minimally Effective teachers who are retained, performance increases on average by 27% of an SD relative to otherwise-similar Developing teachers—quite similar effects to those from the first 3 years of IMPACT.…”

“…Little evidence bears directly on the relationship between teacher evaluation, teacher turnover, and student achievement. A couple of notable exceptions provide a foundation for our research, demonstrating that teacher evaluation can induce less effective teachers to disproportionately exit, raising the quality of teacher effectiveness (Cullen et al, 2019; Dee et al, 2019; Dee & Wyckoff, 2015; Sartain & Steinberg, 2016; Stecher et al, 2018). In a telling example, teacher evaluation reform in Houston increased low-performing teachers’ voluntary exit by 6.2% (Cullen et al, 2019).…”

“…First, most of the lowest performing DCPS teachers at the inception of IMPACT have now exited—voluntarily or involuntarily. Second, many teachers whose scores would previously have designated them as Effective now fall in the Developing range; as such, they face a credible dismissal threat, which increases voluntary attrition (Dee et al, 2019). Third, changes in 2012–2013 to IMPACT’s financial incentives for Highly Effective teachers concentrated incentives in high-poverty schools, which may increase turnover for the district’s most effective teachers, who had been disproportionately situated in low-poverty schools.…”