Cultures of evaluation: tales from the end of the line

Abstract

Experiences in and around aid agencies suggest that the results agenda militates against a culture of learning and improvement to which evaluations should usefully contribute. In this article, I argue that this is an issue with both ethical and operational dimensions. The reductionist and simplifying effects of quantitative indicators of achievement as instruments of performance management, which I see as characteristic of audit culture, are having a pernicious effect in many aid environments. The technical and instrumental ways monitoring and evaluation is understood within the everyday culture of aid agencies preclude the space and time necessary for an ethically premised culture of learning. These social effects of audit culture detract from our capacity as a sector to deliver positive change, and should be of concern to evaluators, commissioners and consumers of evaluations

Participant voice first: the Busara research agenda on ethical research in the Global South

“…The scope of these debates about how to conduct research ethically go well beyond those usually encompassed by the term 'research ethics' (Herington & Tanona, 2020). We need to move beyond a narrow focus only on the welfare of individual participants (Camfield & Palmer-Jones, 2015), though of course that remains crucial too, and examine our role in a wider ecosystem of international development knowledge, especially when our research findings are likely to have an impact on policy and programs Scott, 2016).…”

“…Where the data supports this, these will be standardised protocols that can work well across multiple communities. We will work to ensure that these are practical approaches that take account of the many pressures faced by researchers that make it harder to practice ethical research (Scott, 2016). Wherever possible, we will suggest improvements that can be incorporated into remote research methods.…”

Why work “at home”? Oxfam’s value-added and the UK Poverty Programme

“…Firstly, there are questions as to whether civil society should adopt business practices because they are unsuited to tackling issues of social injustice and inequality. Secondly, there is evidence that the 'audit culture' inherent in results-based evaluation practices is detrimental to learning in aid agencies (Scott 2016). The growth of the use of social audits in the corporate and NGO sectors is one explanation of the increasing concern with value-added.…”

Audit as Genre, Migration Industries, and Neoliberalism’s Uptakes

“…Language scholars (De Costa et al 2019;Morris, 2017;O'Regan and Gray, 2018) and beyond (Kipnis, 2008;Scott, 2016;Shore andWright, 1999, 2015;Vannier, 2010;Welch, 2016) have argued that this proliferation of auditing in all sectors of social and professional life, not only in for-profit corporate organizations, but also in cooperatives, charities, associations, and institutions of the public sector, is emblematic of neoliberal governmentality. This is a contemporary mode of ruling that subjects every domain of social life to market-type rationality (Collier, 2005).…”