A Review of Recent Research on Individual‐Level Score Reports

Abstract

As the primary interface between test developers and multiple educational stakeholders, score reports are a critical component to the success (or failure) of any assessment program. The purpose of this review is to document recent research on individual‐level score reporting to advance the research and practice of score reporting. We conducted a search for research studies published or presented between 2005 and 2015, examining 60 scholarly works for (1) the research focus, (2) stated or implied theoretical frameworks of communication, and (3) the characteristics of data sets employed in the studies. Results show that research on score properties, especially subscores, and score report design/layout are well‐represented in the literature base. The predominant approach to score reporting has been through a cybernetics tradition of communication. Data sets were often small or localized to a single context. We present example research questions from novel communication frameworks, and encourage our colleagues to adopt new roles in their relationships to stakeholders to advance score reporting research and practice.

Examining Score Report Language in Accountability Testing

“…Guidelines such as minimizing the use of jargon and presenting scores in both graphical and narrative forms have been promoted (Goodman and Hambleton, 2004;Zenisky and Hambleton, 2015) in the context of maximizing appropriate interpretations. This orientation to score reporting aligns with an information-processing or cybernetics perspective on communication (Behrens et al, 2013;Gotch and Roduta Roberts, 2018). However, when viewing displays of information through a semiotics communication lens, the choice of words and visual presentation serve to establish tone, and communicate the relative importance of certain types of information.…”

“…Accordingly, most advances to-date have focused primarily on methods for presenting information about test performance in ways that maximize accurate interpretation (e.g., dynamic reporting elements, Zapata-Rivera and VanWinkle, 2010; online tutorials, Zapata-Rivera et al, 2015). In line with this validity-oriented emphasis, current approaches to the communication of assessment results primarily focus on the transmission and processing of information by the end users (Behrens et al, 2013;Gotch and Roduta Roberts, 2018). In this frame of communication, information flows from the test developer to the user, and an effective transmission has taken place when the information is understood by the user in accordance with the intentions of the test developer.…”

Development and Examination of a Tool to Assess Score Report Quality

“…Further, we have situated the rating scale in the specified logic model, which posits causal connections between report content, and outcomes such as interpretation. Therefore, we suggest the rating scale best represents the cybernetics tradition within communications (Craig, 1999;Craig and Muller, 2007;Gotch and Roduta Roberts, 2018). The cybernetics communication lens, or information processing layer as described by Behrens et al (2013), can be described as focused on the transmission of technical information with the goal of minimal distortion upon reaching the receiver.…”

“…The current research base, however, cannot address with confidence the assumption that higher quality reports relate to more positive outcomes. A recent review of the literature (Gotch and Roduta Roberts, 2018) documents fundamental issues in score report research, among them reliance upon small samples bounded within a particular context. Additionally, approaches to assessing score report quality have been indirect, often relying on qualitative user feedback.…”

Interpreting Reports From Universal Screeners: Roadblocks, Solutions, and Implications for Designing Score Reports

“…However, a review of recent research by Gotch and Roberts (2018) showed that between 2005 and 2015, there were only 60 scholarly works retrieved on score reports, with only nine examining the relationship between design/layout and enduser interpretations.…”