Evaluating Special Education Teachers’ Classroom Performance: Rater Reliability at the Rubric Item Level

Abstract

Classroom observations are an integral component of teacher evaluation systems, but little is known about who is best qualified to observe and evaluate special educators, who have a specialized skillset, and whether observation instruments adequately reflect their instructional practices. In this study, 19 special education teachers in California and Idaho each contributed three video-recorded classroom lessons. Using rubric items designed to reflect efficacious instructional practices for teaching students with disabilities, school administrators and peers scored the teachers’ lessons. Rater reliability and sources of error variance were examined using generalizability theory. Findings indicate that peers were more reliable raters than school administrators, who did not have expertise in special education, and the school administrators’ ratings varied at the rubric items level. Implications for classroom observation systems are discussed.

Preparing Special Educators and School Principals in Special Education Law: An Undervalued Area of Professional Training?

“…In effect, programs providing a school law course have separated special education law from day-to-day principal concerns, even as principals-in-training reported not receiving such courses (Angelle & Bilton, 2009; McHatton et al, 2010). Although general texts for principals do commonly mention personnel development, the lack of broad special education content is concerning in that principals seem underprepared to evaluate special educators (Lawson & Cruz, 2017; Lawson & Knollman, 2017; Rodl et al, 2018) as one of their primary administrative roles. Given the long-standing strain on schools to recruit and retain special educators (Billingsley & Bettini, 2019), and the link between administrative support and exiting the field, it is imperative that principals are prepared to support the special educators in their building.…”

Policies That Define Instruction: A Systematic Review of States’ and Districts’ Recommendations for Evaluating Special Educators

“…Preobservation conferences are promoted or required in many evaluation systems (Steinberg & Donaldson, 2016), but there is evidence that they may not take place due to time constraints (Kraft & Gilmour, 2016). Additionally, even if school leaders use preobservation conferences to customize evaluation systems, evaluators without special education experience struggle with evaluating SETs, primarily SETs’ use of instructional practices aligned with effective instruction for SWDs (Lawson & Cruz, 2018a, 2018b). Flexibility resulting in effective and appropriate modifications requires a knowledgeable evaluator, a knowledgeable SET, and a cooperative relationship between the evaluator and the SET.…”