Educational Standards, Assessment, and the Search for Consensus

Abstract

In this article, we critically examine the nature of the "consensus" reflected in educational standards used to orient high-stakes assessment programs. We analyze two complementary cases of practice in the assessment of teaching. One focuses on the discourse of standards creation and one examines how standards like these are typically used to orient assessment development and judgments about individual performance. We offer two (partially competing) theoretical perspectives that might illuminate and guide our practices in this currently undertheorized and underexamined area of standards development. One is based in the discourse ethics of Jü rgen Habermas and one is based in critical elaborations of Hans-Georg Gadamer's philosophical hermeneutics. We argue that conventional consensus-seeking approaches to the development and public review of educational standards tend to mask diversity and relinquish authority for consequential decisions to assessment developers who work in far less public circumstances. We draw on hermeneutic philosophy to offer a more pluralist approach that allows dissensus to be represented and taken into account in the assessment process.

Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria

“…In particular, assessors may not understand the outcomes they are supposed to be judging (Baume, Yorke and Coffey 2004), may not agree with, ignore or choose not to adopt the criteria (Baume, Yorke and Coffey 2004;Ecclestone 2001;Orrell 2008;Smith and Coombe 2006) or interpret them differently (Webster, Pepper and Jenkins 2000). The language of criteria or standards can mask differences in interpretation (Moss and Schutz 2001). Studies have found that experienced markers are no better than novice markers at applying standards consistently.…”

Controversies of Standardized Assessment in School Accountability Reform: A Critical Synthesis of Multidisciplinary Research Evidence

“…Such a participatory process (Haertel, 2002) not only improves teacher em-CONTROVERSIES OF STANDARDIZED ASSESSMENT powerment by reducing threats to teacher autonomy, it also helps ensure that the content standards and passing standards are challenging and yet attainable by all, if indeed such an assessment goal is supported by theory and feasible in practice. Consensus-seeking procedures that build on the strength of diversity in the standards development process have been proposed by Moss and Schutz (2001). Democratic evaluation that examines power relations in the evaluation context by including different stakeholders' perspectives might be the answer to addressing the concerns over a top-down hierarchical approach to accountability (Ryan, 2004).…”

Rearticulating Writing Assessment for Teaching and Learning

“…In a description of a training session, Robert Broad (1994) illustrates the ways in which interpretation and meaning are predetermined through the use of anchor papers and training rubrics, so that readers are compelled to read student writing from one specific point of view. More recently, Pamela Moss and Aaron Schutz (2001) illustrate the same phenomenon in describing the creation of standards for teacher certification. This way of reading is antithetical to the kinds of meaning making and interpretation that most often accompanies the way in which people come to value certain kinds of texts (Smith 1988).…”