‘Catalyst data’: perverse systemic effects of audit and accountability in Australian schooling

Abstract

This paper examines the perverse effects of the new accountability regime central to the Labor government's national reform agenda in schooling. The focus is on National Assessment Program -Literacy and Numeracy (NAPLAN) results that now act as 'catalyst data' and are pivotal to school and system accountability. We offer a case study, with two embedded units of analysis, in which NAPLAN has become high stakes testing for systems. The first involves the relationships between the federal government and three States (Victoria, New South Wales and Queensland) in negotiating performance targets on NAPLAN for reward payments in respect of a national agreement to improve literacy and numeracy. We show how Victoria used 2009 data as baseline, set ambitious targets and failed to meet them, while Queensland set much less ambitious targets, met them and was rewarded. New South Wales created targets that combined literacy and numeracy scores, obfuscating the evidence, and met their targets. The second focuses specifically on Queensland and the ramifications of the poor performance of the State on the 2008 NAPLAN. This resulted in a review commissioned by the Premier, a Report on how to improve performance, and the introduction of Teaching and Learning Audits and State-wide targets for improvement on NAPLAN. This unit of analysis focuses on the perverse effects of this highly politicized agenda. This paper shows how States seek to protect their 'reputational capital' and as such, 'game' the system. The data for the analysis draw upon interviews with relevant senior policy-makers and on analysis of relevant documents and media coverage.

A logic of enumeration: the nature and effects of national literacy and numeracy testing in Australia

“…schools throughout the country. NAPLAN was not only a catalyst for an increased focus upon performativity at a system level, with attendant concerns about 'reputational capital' at this level (Lingard & Sellar, 2013), but also a catalyst for similar concerns at the school level itself.…”

“…It is also a reflection of the continued need for assessment literacy within the field, and that this needs to be a continuing part of teachers' growth and development if it is to enable productive challenging of current assessment practices (Klenowski, 2011;Sebba, 2006 when published constitute a form of objectified, and also institutionalised capital, seemed to shift the curricula focus from an emphasis upon learning for its own sake, to a focus upon securing the best possible test scores as markers of esteem, and as proxies of learning (particularly in comparison with 'like' schools). Such capitals were 'possessed', or perhaps more accurately, 'possessed' all to whom they were directed -teachers striving to teach to effect improvements in such results, school administrators seeking to have their schools represented in the best light possible in public fora such as MySchool, and those students being 'pushed' by their teachers to do as well as possible to safeguard the 'reputational capital' attached to these results (Lingard & Sellar, 2013). Various reading groups and reading strategies were seen as likely to contribute to improved NAPLAN results.…”

“…These quantification processes also have a tendency to have the most substantive effects in those arenas most susceptible to external influence (Porter, 1995). This emphasis upon testing is often construed in response to broader competitive and comparative pressures within and between nation-states about concerns about national competitiveness under current global conditions (Stobart, 1998;Lingard & Sellar, 2013). Under these circumstances, the effects of such strong policy support for testing upon more educative processes and practices is an area of potential concern.…”

“…Perverse effects of standardised testing have been noted systemically, including the way in which different states in Australia have sought to represent data in particular ways to ensure accountability benchmarks are most likely to be met so as to secure federal government reward payments. Lingard and Sellar (2013) draw, inter alia, upon theoretical resources including neoliberalism, governmentality and Deleuzian notions of control societies, to refer to how personnel in the state of Victoria set ambitious targets for NAPLAN results, and were unsuccessful in attaining them, while New South Wales combined literacy and numeracy targets as a means of potentially shielding problematic year levels/elements from auditors, and succeeded in meeting their targets. Queensland, which performed most poorly relative to the other two states in earlier tests, set relatively lower targets, and succeeded in achieving these.…”

Education policy-making and time

“…One manifestation of this interiority is the emergence of standardised testing as "the chief instrument of educational governance" (Trohler, 2010, p. 6). The rise of testing through policymaking has been described in a number of ways, ranging from Ball's (2003, p. 223)"performative culture", Lingard and Sellar's (2013) "catalyst data", to what Ozga has described as the shift from "government to governance" as evidenced by the growth in data as a "policy instrument" (2009, p. 150). As Ozga has suggested, this shift has occurred progressively over time.…”