The Cbal Reading Assessment: An Approach for Balancing Measurement and Learning Goals

Abstract

No Child Left Behind has highlighted the need for new types of assessments that not only provide high-quality evidence about what students know and can do, but also help to move learning forward. This paper describes a linked set of formative and summative reading assessments designed to address the tradeoffs inherent in these two goals.Targeted skills include the full range of competencies underlying proficient reading at the middle-school level, including both lower-order skills such as oral reading fluency and decoding, and higher-order skills such as the ability to integrate and synthesize information from multiple texts. Data collected in pilot administrations of two prototype test forms are presented. Analyses suggest that this new approach yields acceptable measurement properties while simultaneously addressing crucial learning outcomes. This ii Table 4 CBAL assessments seek to achieve these goals by incorporating the following innovations: (a) Linked summative and formative assessments are based on a theory of domain competency that specifies both what students should be learning and how that learning is likely to develop over time; (b) assessments include innovative scenario-based tasks designed to model expert teaching practice and to encourage the use of classroom activities that have been shown to support learning; (c) assessments are administered at multiple time points spaced throughout the school year so that information about student achievement can be shared with teachers while there is still time to take needed instructional action; and (d) state-of-the-art automated scoring technologies are used to broaden the array of skills assessed, and to ensure that score reports are provided in a timely manner. List of TablesAlthough the CBAL Initiative includes summative, formative, and professional support components, this paper addresses the summative and formative components only. In particular, we report research focused on the development and evaluation of linked summative and formative reading assessments designed to provide high quality evidence for state accountability purposes while simultaneously addressing key learning goals. . The remainder of this paper is structured as follows. First, we describe the framework developed to guide the assessment design; second, we describe a set of prototype reading assessments targeted at readers in Grades 7 and 8; and finally, we present pilot data collected in a large northeastern school district. 2 The CBAL Reading FrameworkExisting accountability assessments have been characterized as representing a view of proficiency that is "a mile wide and an inch deep" (Schmidt, McKnight, & Raizen, 1997). CBAL assessments, by contrast, are designed to collect deeper evidence about a more modest number of instructionally-relevant competencies. The framework developed to guide this process includes three structures: a competency model, a set of hypothesized learning progressions (LPs), and a set of task design principles. These structures are described be...

Cbal: Results From Piloting Innovative K–12 Assessments

“…Although dimensionality results suggest that the comprehension test forms might measure a single dimension, Sheehan and O'Reilly (2009) presented results that lend partial support to the distinction between model-building and applied-comprehension skills depicted in the CBAL reading competency model. The distinction supported is not of qualitatively different skills but, rather, of differences in difficulty between items written to target the two hypothesized skill types.…”

A Historical Survey of Research Regarding Constructed-Response Formats

“…The process entailed the formulation of competency models for specific domains, including reading (Sheehan and O'Reilly 2011), writing , mathematics (Graf 2009), and science (Liu et al 2013); the elaboration of constructs, especially writing ; and innovations in automated scoring (Deane 2013a, b;Fife 2013) and task design (Bennett 2011;Sheehan and O'Reilly 2011).…”

Preliminary Reading Literacy Assessment Framework: Foundation and Rationale for Assessment and System Design

“…While existing K-12 summative assessments are generally considered useful for certain purposes, such as rankordering students on a unidimensional scale of reading ability or providing comparative information across groups and in relation to national norms, they have been repeatedly criticized on a number of other counts. These criticisms include, but are not limited to, a narrow or outdated view of the construct, the lack of diagnostic information or inactionable data (e.g., scores given at the end of year when there is no time to adjust instruction), ineffective score reports, weak links to instruction, scant measurement of partial knowledge at the lower end of the distribution, and inadequate tailoring of the assessment to the individual needs of the student Bennett & Gitomer, 2009;Bransford et al, 2000;Chou et al, 2007;Cromley & Azevedo, 2007;Cutting & Scarborough, 2006Francis et al, 2006;Glaser & Silver, 1994;Graesser & Hu, 2012;Guterman, 2002;Hannon & Daneman, 2001;Katz & Lautenschlager, 2001;Keenan, 2012;Magliano et al, 2007;Moran et al, 2008;O'Reilly & Sheehan, 2009;Partnership for 21st Century Skills, 2004;Pellegrino et al, 2001;Rupp et al, 2006;Sheehan & O'Reilly, 2011).…”

“…The second type of assessment is intended to measure global 11 reading literacy ability (see Bennett, 2011;Bennett & Gitomer, 2009;Deane et al, 2012;O'Reilly et al, 2012;O'Reilly & Sheehan, 2009;Sabatini, Bennett, & Deane, 2011;Sheehan & O'Reilly, 2011. More specifically, we refer to this assessment as the Global Integrated Scenario-Based Assessment of reading, or GISA for short.…”

“…A fourth way to infer developmental or sophistication levels is to vary the use of support for a given task (see, Bennett, 2011;Bennett & Gitomer, 2009;Deane et al, 2012;O'Reilly & Sheehan, 2009;Sheehan & O'Reilly, 2011Vygotsky, 1978). Students at lower levels of development can be given textual aids (audio, video, vocabulary definitions) to help them perform a task targeting a specific proficiency.…”