Between Scylla and Charybdis: Reflections on and problems associated with the evaluation of teachers in an era of metrification

Abstract

Abstract:The Scylla and Charybdis in this discussion of teacher evaluation are standardized achievement test data on the one hand, and classroom observational systems on the other. These are the two most common methods used to judge teachers' competency. Both have serious flaws: the former primarily with validity, the latter primarily with reliability. At most these evaluation strategies provide teachers' and their supervisors information about which to converse. But these two methods have such serious flaws that they should never be used as the primary grounds for rewarding, punishing, or firing teachers. When both methods of evaluation are used to judge teacher competency, the correlation between achievement tests and observational data is quite low. When two methods claiming to assess the same construct do not correlate well, either one or both Reflexiones sobre problemas asociados con la evaluación de maestros en una época de mediciones Resumen: En esta discusión de la evaluación de los maestros Escila y Caribdis representan, por un lado, las pruebas estandarizadas de desempeño y, por el otro, los sistemas de observación de clase. Estos son los dos métodos más comunes para juzgar la competencia de los maestros. Ambos tienen serias deficiencias; el primero con su validez, el segundo básicamente con su confiabilidad. En el mejor de los casos, estas estrategias de evaluación ofrecen a los maestros y sus supervisores temas de conversación; pero ambos tienen serias deficiencias y no deberían usarse en ningún caso como fundamentos para premiar, castigar o despedir a los maestros. Cuando ambos métodos se usan para juzgar la competencia de los maestros, la correlación entre pruebas de desempeño y datos derivados de la observación es muy baja. Cuando los dos métodos aseguran evaluar el mismo constructo, no se correlacionan bien; uno o ambos fracasan al evaluar el constructo pretendido. Hay dos alternativas para navegar entre Escila y Caribdis: la "Evaluación basada en las obligaciones de los maestros" y las "medidas de desempeño". Estos métodos son muy recomendables, aunque como todo método de evaluación personal, la confiabilidad y la validez siguen siendo asuntos de discusión muy problemáticos. Palabras-clave: Evaluación de maestros; malos maestros; pruebas estandarizadas de logro; instrumentos de observación; observación en clase; validación de constructos; evaluación basada en las obligaciones de los maestrosReflexões sobre problemas associados à avaliação de professores em um momento de medições Resumo: Nesta discussão da avaliação dos professores Scylla e Charybdis, eles representam, por um lado, os testes de desempenho padronizados e, por outro, os sistemas de observação de classe. Estes são os dois métodos mais comuns para julgar a competência dos professores. Ambos têm sérias deficiências; o primeiro com sua validade, o segundo basicamente com sua confiabilidade. Na melhor das hipóteses, essas estratégias de avaliação oferecem aos professores e seus supervisores tópicos de conversação; mas ambos têm sérias d...

Governing teachers through datafication: Physical–virtual hybridity and language interoperability in teacher accountability

“…problems associated with validity, reliability and bias), as well as the negative consequences stemming from their use in teacher evaluation policies (e.g. narrowing of the curriculum and teaching to the test, see, for example, Amrein-Beardsley 2014; Berliner 2018). In Texas specifically, VAMs have been a contentious issue that has been taken up in policy debates and the courts (Amrein-Beardsley and Collins, 2012).…”

The value of student feedback in open forums: A natural analysis of descriptions of poorly rated teachers

“…Most of these are unplanned and unpredictable and the thoughts that are behind them are typically unknowable. Choosing between the two most common and equally flawed evaluation systems (test-based and observational evaluations) is akin to being between Scylla and Charybdis (Berliner, 2018). Problem-free teacher evaluation systems do not exist.…”

Digital Disruption In Teaching And Testing

“…The third component, student growth measurement, requires districts to develop an appropriate system for determining student achievement growth, and this can be in the form of portfolios, student learning objects (SLOs), or value-added measurements (VAM). VAM, the most controversial approachbut arguably the most statistically sophisticated (Berliner, 2018) has been used widely across Texas (and the US more broadly) for decades. It is designed to measure student growth from one year to the next, on state-wide standardized achievement tests, and link that growth to their teacher.…”