Exploring the dimensionality of self-perceived performance assessment literacy (PAL)

Abstract

Performance assessments attempt to provide a practical and authentic demonstration of students' learning. Despite growing investments in performance assessments by states, as well as researchers' theorized value of this type of assessment, the field has not developed a measure of assessment literacy specific to performance assessments that has sufficient psychometric evidence to support it. This study begins important research on developing a quantitative measure that can be used by educational practitioners to self-evaluate their own performance assessment literacy (PAL). Using the Quality Performance Assessment (QPA) framework from the [organization masked for blind review] as a foundation, this study explores and confirms the dimensionality of a 27-item survey instrument that assesses educational practitioners' perceptions of their PAL using exploratory and confirmatory factor analysis. Our findings provide evidence that the instrument captures five reliable dimensions of PAL: valid design, reliable scoring, data analysis, fair assessment, and student voice and choice.

A review of teachers’ assessment literacy proficiency measures

“…The second type is the Likert-scale questionnaire. Five questionnaires are listed: Assessment Literacy Survey (ALS) [26], Questionnaire for Assessment Literacy (QAL) [27], Approaches to Classroom Assessment Instrument (ACAI) [28], English Language Teachers' Assessment Literacy (ELTAL) [29], and Performance Assessment Literacy (PAL) [1]. In the last category, the survey with the inclusion of both test and questionnaire is labeled as the Questionnaire for Measurement Literacy (QML) [30].…”

“…Rather than measuring AL in general, a study [1] explored and confirmed a quantitative measure for teachers to self-evaluate their proficiency level on performance AL. The 27-item instrument was framed by five dimensions identified in the Quality Performance Assessment Framework by Brown and Mednick [34] as essential elements of performance assessment practices: valid assessment design, reliable scoring, Int J Eval & Res Educ ISSN: 2252-8822  assessment data analysis, fairness in assessment, and student voice and choice.…”

“…Assessment literacy (AL) is a crucial element of teachers' professional development [1], [2] and it is necessary for teachers to have an adequate level in AL to realize the full potential of assessment in the teaching and learning process [3]- [6]. Regrettably, the existing studies suggest that teachers' AL is insufficient to meet the assessment requirements advocated in assessment policies or standards, even the excellent teachers feel challenged to prepare assessment [7]- [9].…”

Untitled

“…En este caso, se identifican aspectos alusivos a la evaluación socialmente justa (transparencia, uso de pruebas cualitativas junto al examen, evaluación de actitudes y valores), dando cuenta de la multidimensionalidad del concepto. En complemento, Kelly et al (2020) incorporan antecedentes históricos sobre la evaluación de desempeño docente y proponen una escala Likert para evaluar cinco dimensiones claves de la alfabetización en evaluación: diseño válido, calificación confiable, análisis de datos, evaluación justa, voz y elección de los estudiantes. Desde nuestro punto de vista, lo interesante de estas investigaciones es que exponen con claridad algunos marcos conceptuales ligados a visiones democratizadoras o más integrales de la educación, que podrían incidir en la (re)elaboración de las representaciones acerca de la enseñanza, el aprendizaje y la evaluación, en coherencia con la expectativa de avanzar hacia una mirada pedagógica del proceso evaluativo (MINEDUC, 2018).…”

A Content Validity Study for Vocational Teachers’ Assessment Literacy Instrument (VoTAL)

“…Classroom assessment has been described as the cornerstone of the current education system (Brookhart, 2011;Kelly et al, 2020;Popham, 2014). Previous researches had emphasized the significance of assessment in strengthening the education system (Birenbaum et al, 2015;Brookhart, 2011;DeLuca, 2012;Popham, 2014).…”