Disability Data Collection in a Complex Humanitarian Organisation: Lessons from a Realist Evaluation

Abstract

In recent years, global attention to disability inclusion in humanitarian and development contexts, notably comprising disability inclusion within the Sustainable Development Goals, has significantly increased. As a result, UN agencies and programmes are increasingly seeking to understand and increase the extent to which persons with disabilities are accounted for and included in their efforts to provide life-saving assistance. To explore the effects and effectiveness of such measurement, this paper applies a complexity-informed, realist evaluation methodology to a case study of a single measurement intervention. This intervention, ‘A9’, was the first indicator designed to measure the number of persons with disabilities assisted annually by the United Nations World Food Programme (WFP). Realist logic of analysis combined with complexity theory was employed to generate context-mechanism-outcome configurations (CMOC’s) against which primary interviews and secondary data were analysed. We show that within the complexity of the WFP system, the roll-out of the A9 measurement intervention generated delayed, counter-intuitive and unanticipated effects. In turn, path dependency and emergent behaviours meant that the intervention mechanisms of yesterday were destined to become the implementation context of tomorrow. These findings challenge the current reliance on quantitative data within humanitarian-development disability inclusion efforts and contribute to our understanding of how data can best be leveraged to support inclusion in such contexts.