Performance Assessment for Principal Licensure: Evidence From Content and Face Validation and Bias Review

Abstract

This article presents the validity bias review feedback and outcomes of new performance-based assessments to evaluate candidates seeking principal licensure. Until now, there has been little empirical work on performance assessment for principal licensure. One state recently developed a multi-task performance assessment for leaders and has implemented it for statewide use in initial principal licensure decisions; this development process is described here, focusing on content validity and bias review, and incorporates candidate and program faculty validiation as well. The results demonstrate the content validity, relevance, and feasibility of this new performance assessment for leaders, and yield implications for leader assessment generally.

The Performance Assessment for Leaders: Construct Validity and Reliability Evidence

“…The PAL assessments were created by an assessment development team, that included the authors, through a multistep assessment development process described in a related article (Orr et al, 2017). The PAL assessments are made up of the following four tasks (see Appendix A for a description of each): Task 1: Leadership through a vision for high student achievement; Task 2: Instructional leadership for a professional learning culture; Task 3: Leadership in observing, assessing, and supporting individual teacher effectiveness; and Task 4: Leadership for family engagement and community involvement.…”

“…Candidates produce a portfolio of work that includes three or four artifacts for each task, commentary on leadership skills used to complete each task, and supporting documents that inform scoring (the artifacts are summarized in Appendix B and explained in the companion article; Orr, et al, 2017). Candidates can complete and submit their work products at any time but are only scored when each task portfolio is complete.…”

School leadership and educational effectiveness

“…As described by Orr et al (2018) initial principal licensing in the US is also based on candidate's experience (specific number of years of teaching, advanced graduate preparation etc. ), while some states also require candidates to complete a leadership exam (national or a state-designed), or have even introduced a pre-service multi-task performance tests to determine potential school heads' competences (Orr et al, 2018).…”

Performance assessment for school leaders: comparing field trial and implementation results

“…This candidate assessment requirement was implemented simultaneously with revised requirements for leadership preparation for the state’s multiple university and non-university-based pathways. The new requirements were intended to foster the certification of a large number of candidates[1] in a variety of both district settings (from very rural to large city) and leadership preparation pathways: a state-approved preparation program, an administrative apprenticeship/internship option, and a panel review option (Orr, Pecheone, Snyder, Murphy, Palanki, Beaudin et al , 2018).…”

“…PAL, the first performance assessment for school leader licensure in the country, handily meets these assessment quality expectations. Early evaluation of its validity and reliability – from the pilot study on feasibility and relevance to the yearlong, statewide field trial – indicates the promise of PAL (Orr, Pecheone, Hollingworth et al , 2018; Orr, Pecheone, Snyder, Murphy, Palanki, Beaudin et al , 2018). As will be shown below, these results extend to the first year of implementation, characterized by further refinements and the addition of cut scores for consequential licensure decisions.…”