Enacting the Rubric: Teacher Improvements in Windows of High-Stakes Observation

Abstract

Teacher evaluation systems that use in-class observations, particularly in high-stakes settings, are frequently understood as accountability systems intended as non-intrusive measures of teacher quality. Presumably, the evaluation system motivates teachers to improve their practice – an accountability mechanism – and provides actionable feedback for improvement – an information mechanism. No evidence exists, however, establishing the causal link between an evaluation program and daily teacher practices. Importantly, it is unknown how teachers may modify their practice in the time leading up to an unannounced in-class observation, or how they integrate feedback into their practice post-evaluation, a question that fundamentally changes the design and philosophy of teacher evaluation programs. We disentangle these two effects with a unique empirical strategy that exploits random variation in the timing of in-class observations in the Washington, D.C. teacher evaluation program IMPACT. Our key finding is that teachers work to improve during periods in which they are more likely to be observed, and they improve with subsequent evaluations. We interpret this as evidence that both mechanisms are at work, and as a result, policymakers should seriously consider both when designing teacher evaluation systems.

Critical Feedback Characteristics, Teacher Human Capital, and Early-Career Teacher Performance: A Mixed-Methods Analysis

“…1 A growing body of work suggests that the success of these reforms rests partly on teacher observation processes and the policies that shape them (Donaldson, 2021). Specifically, teaching improvements seem to depend on the quality of observers’ training (Steinberg & Sartain, 2015), observers’ discretion in the implementation of teacher observations, structured observation conferences, or post-observation improvement plans (Donaldson & Woulfin, 2018; Hunter & Ege, 2021; Marsh et al, 2017), and teachers’ receipt of observations (Phipps, 2018; Phipps & Wiseman, 2021). Recent quasi-experimental work also indirectly implies that observations improve teachers’ performance by increasing their motivation via accountability mechanisms and performance feedback (Phipps & Wiseman, 2021).…”

“…Specifically, teaching improvements seem to depend on the quality of observers' training (Steinberg & Sartain, 2015), observers' discretion in the implementation of teacher observations, structured observation conferences, or post-observation improvement plans (Donaldson & Woulfin, 2018;Hunter & Ege, 2021;Marsh et al, 2017), and teachers' receipt of observations (Phipps, 2018;Phipps & Wiseman, 2021). Recent quasi-experimental work also indirectly implies that observations improve teachers' performance by increasing their motivation via accountability mechanisms and performance feedback (Phipps & Wiseman, 2021). 2 Although feedback generally improves teacher performance, theoretical and empirical research suggests that certain feedback characteristics are more potent than others.…”

Examining the demands of teacher evaluation: time use, strain and turnover among Tennessee school administrators

“…A small body of quasi-experimental studies has examined reformed evaluation systems' effects on teaching and student achievement outcomes and found mixed results (Donaldson, 2021). Reformed systems do not consistently improve student achievement scores in the same subject (Steinberg and Donaldson, 2016;Garet et al, 2017;Taylor and Tyler, 2012) and these improvements may be short-lived (Steinberg and Donaldson, 2016;Garet et al, 2017), but evaluation seems to improve teacher observation scores (Garet et al, 2017;Phipps and Wiseman, 2020).…”

Not enough hours in the day

“…Philipp and Kunter (2013 [51]) hypothesised that the increase in administrative work among more experienced teachers in Germany might be explained by their desire to reduce more exhausting tasks as they approach retirement age (the proportion of time spent on administrative tasks was associated with lower levels of exhaustion in the German study). It could also reflect the increase in teachers' administrative responsibilities as they take on more managerial positions.…”