The evidence-policy divide: a ‘critical computational linguistics’ approach to the language of 18 health agency CEOs from 9 countries

Abstract

BackgroundThere is an emerging body of literature suggesting that the evidence-practice divide in health policy is complex and multi-factorial but less is known about the processes by which health policy-makers use evidence and their views about the specific features of useful evidence. This study aimed to contribute to understandings of how the most influential health policy-makers view useful evidence, in ways that help explore and question how the evidence-policy divide is understood and what research might be supported to help overcome this divide.MethodsA purposeful sample of 18 national and state health agency CEOs from 9 countries was obtained. Participants were interviewed using open-ended questions that asked them to define specific features of useful evidence. The analysis involved two main approaches 1)quantitative mapping of interview transcripts using Bayesian-based computational linguistics software 2)qualitative critical discourse analysis to explore the nuances of language extracts so identified.ResultsThe decision-making, conclusions-oriented world of policy-making is constructed separately, but not exclusively, by policy-makers from the world of research. Research is not so much devalued by them as described as too technical— yet at the same time not methodologically complex enough to engage with localised policy-making contexts. It is not that policy-makers are negative about academics or universities, it is that they struggle to find complexity-oriented methodologies for understanding their stakeholder communities and improving systems. They did not describe themselves as having a more positive role in solving this challenge than academics.ConclusionsThese interviews do not support simplistic definitions of policy-makers and researchers as coming from two irreconcilable worlds. They suggest that qualitative and quantitative research is valued by policy-makers but that to be policy-relevant health research may need to focus on building complexity-oriented research methods for local community health and service development. Researchers may also need to better explain and develop the policy-relevance of large statistical generalisable research designs. Policy-makers and public health researchers wanting to serve local community needs may need to be more proactive about questioning whether the dominant definitions of research quality and the research funding levers that drive university research production are appropriately inclusive of excellence in such policy-relevant research.

A comparative analysis of policies addressing rural oral health in eight English-speaking OECD countries

“…Critical discourse analysis is an application that allows researchers to examine how language works in subtle ways to reproduce or naturalise certain assumptions and make them seem unquestioned 50 . For this study, a discourse was defined as a distinct set of language (or block of text) that represented assumptions about causality versus language about policy solutions 51 . When discourses included both content about causality and solutions, the blocks of text were further analysed to determine an individual focus.…”

Decision-making experiences of public library CEOs

“…Within the public library management population, investigation into the link between self-confidence, and intuition should be explored. Some overlap is noted in considering whether public library CEOs work in the context of an evidence-policy divide for library, such as in the health and education sectors (Bell and Seidel, 2012;Favero and Meier, 2013;Moynihan and Ingraham, 2010). In addition, the data collected from this study could inform additional work with non-Canadian public library CEOs and a longer period of monitoring to follow a full fiscal year could reveal new insights.…”

Understanding the Research–Policy Divide for Oral Health Inequality

“…In our previous study (Bell and Seidel, 2012), we explored how disadvantaged groups are represented in oral health policy statements from Organisation for Economic Co-operation and Development (OECD) countries. We found that concepts for "vulnerable or disadvantaged groups are usually, but not always, the least frequent concepts" in these documents.…”

“…These oral health policy documents, like the abstracts, were treated as indicative evidence, i.e., of oral health policy understandings, not necessarily what has been implemented. Our previous study of the representation of disadvantaged groups in oral health policy (Bell and Seidel, 2012) used criteria from other international comparative studies to define a policy statement as follows:…”

“…It is known that the researchpolicy divide in health has a complex causality (Nutley et al 2007): policy making is aligned to political, financial and strategic imperatives (Corrigan and Watson 2003;Lavis et al 2001); evidence does not always capture different stakeholder needs (Lavis et al 2002); and many "real-world" local contextual limitations unaccounted-for by researchers apply to policy-making (Innvaer et al 2002). Analysis of the literature explores debates about how quantitatively defined disciplines such as oral health too often fail to capture the complex contexts of policy, while qualitative research is perceived as lacking the defensible rigour required for the adversarial and warring interests found in policy contexts (Bell 2010;Bell and Seidel 2012).…”