Examining Score Report Language in Accountability Testing

Abstract

The purpose of this study was to examine patterns of language use in score reports within a North American context. Using a discourse analysis approach informed by conversation analysis, we explored how language was structured to express ideas or (re-)produce values, practices, and institutions in society. A sample of 10 reports from the United States and Canada within the domain of accountability testing was selected. Observed patterns of language represented micro-discourses embedded within broader discourse related to accountability mandates within each country. Three broad themes were identified within and across the score reports-Displays of Information, Knowledge Claims, and Doing Accountability. Within each of the broad themes were sub-themes-word choice, and visual representations, script formulations, hedging, establishing authority, and establishing responsibility-that characterized more fine-grained textual features. Future research may explore empirical evidence for the social dynamics identified through this study's textual analysis. Complementary lines of research on cognitive, affective, and socio-cultural factors of score report interpretation and use are encouraged.

Principals’ and Teachers’ Comprehension of School Performance Feedback Reports. Exploring Misconceptions from a User Validity Perspective

“…For instance, the amount of specialized and statistical vocabulary to use is a critical consideration (Shivraj & Ketterlin-Geller, 2019) as narrative elements can be too lengthy, too succinct, or otherwise confusing (Hambleton & Slater, 1997). Jargon can be unfamiliar and sometimes intimidating to SPF users, but at the same time vocabulary can also establish tone and authority (Fjørtoft & Lai, 2021;Roduta Roberts et al, 2018). In some cases, supportive information and tutorials can provide guidance (Zapata-Rivera et al, 2016).…”

“…This question is rooted in an information-processing paradigm where providers are senders and users are receivers (Ryan, 2006). Our second research question is inspired by a semiotic paradigm and shifts from a mere sender-receiver outlook to a perspective in which SPF reports are seen as communicative tools between providers and recipients Roduta Roberts, Gotch, & Lester, 2018). How can we explain educational professionals' misconceptions when interpreting SPF?…”