Integrating quasi-experimental and inductive designs in evaluation: A case study of the impact of free bus travel on public health

Green, John; Roberts, Helen; Petticrew, Mark; Steinbach, Rebecca; Goodman, Anna; Jones, Alasdair; Edwards, Phil

doi:10.1177/1356389015605205

Cited by 25 publications

(24 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dataset comprised 278 anonymized interview and focus group transcripts pooled from five UK‐based research studies. Study contexts ranged from commuting in Cambridge, cycling in London and free bus passes for young people in London, to the impact of a new motorway in Glasgow and a graduated drivers license scheme in Northern Ireland . The studies included participants of various ages and gender and represented rural and urban locations across the United Kingdom.…”

Section: Case Study: Applying Leximancer To Synthesize Qualitative Trmentioning

confidence: 99%

Semiautomated text analytics for qualitative data synthesis

Haynes

Garside

Green

et al. 2019

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

Approaches to synthesizing qualitative data have, to date, largely focused on integrating the findings from published reports. However, developments in text mining software offer the potential for efficient analysis of large pooled primary qualitative datasets. This case study aimed to (a) provide a step‐by‐step guide to using one software application, Leximancer, and (b) interrogate opportunities and limitations of the software for qualitative data synthesis. We applied Leximancer v4.5 to a pool of five qualitative, UK‐based studies on transportation such as walking, cycling, and driving, and displayed the findings of the automated content analysis as intertopic distance maps. Leximancer enabled us to “zoom out” to familiarize ourselves with, and gain a broad perspective of, the pooled data. It indicated which studies clustered around dominant topics such as “people.” The software also enabled us to “zoom in” to narrow the perspective to specific subgroups and lines of enquiry. For example, “people” featured in men's and women's narratives but were talked about differently, with men mentioning “kids” and “old,” whereas women mentioned “things” and “stuff.” The approach provided us with a fresh lens for the initial inductive step in the analysis process and could guide further exploration. The limitations of using Leximancer were the substantial data preparation time involved and the contextual knowledge required from the researcher to turn lines of inquiry into meaningful insights. In summary, Leximancer is a useful tool for contributing to qualitative data synthesis, facilitating comprehensive and transparent data coding but can only inform, not replace, researcher‐led interpretive work.

show abstract

Section: Case Study: Applying Leximancer To Synthesize Qualitative Trmentioning

confidence: 99%

Semiautomated text analytics for qualitative data synthesis

Haynes

Garside

Green

et al. 2019

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, if such context is viewed as exemplifying wider social and political forces, the single case can be ‘telling’, rather than ‘typical’, and offer insight into a wider issue [ 34 ]. Internal comparisons within the case can offer rich possibilities for logical inferences about causation [ 17 ]. Further, case studies of any size can be used for theory testing through refutation [ 22 ].…”

Section: Main Bodymentioning

confidence: 99%

“…Case study research, currently under-utilised in public health and health services evaluation, can offer considerable potential for strengthening faith in both external and internal validity. For example, in an empirical case study of how the policy of free bus travel had specific health effects in London, UK, a quasi-experimental evaluation (led by JG) identified how important aspects of context (a good public transport system) and intervention (that it was universal) were necessary conditions for the observed effects, thus providing useful, actionable evidence for decision-makers in other contexts [ 17 ].…”

Section: Introductionmentioning

confidence: 99%

Case study research for better evaluations of complex interventions: rationale and challenges

Paparini

Green

Papoutsi

et al. 2020

BMC Med

Self Cite

View full text Add to dashboard Cite

Background The need for better methods for evaluation in health research has been widely recognised. The ‘complexity turn’ has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether, and under which conditions, interventions in complex systems improve health services or the public health, and what mechanisms might link interventions and outcomes. We argue that case study research—currently denigrated as poor evidence—is an under-utilised resource for not only providing evidence about context and transferability, but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear. Main body Case study research, as an overall approach, is based on in-depth explorations of complex phenomena in their natural, or real-life, settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity, but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However, there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second, there is little consensus about what a case study is, and considerable diversity in how empirical case studies are conducted and reported. Finally, as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail), it can be challenging to identify the key messages related to intervention evaluation from case study reports. Conclusion Whilst the diversity of published case studies in health services and public health research is rich and productive, we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences.

show abstract

“…Leatherdale [18] provides some other suggestions for facilitating greater evaluation evidence of health policies, suggesting the need to improve the ability to evaluate government policy. For this to happen, capacity needs to be built around practitioners either to conduct natural experiments or to work closely with academics so that more robust quasi-experimental methods of evaluation can be employed [76].…”

Section: Strengths and Weaknesses Of Natural Experimentsmentioning

confidence: 99%

Using natural experiments to improve public health evidence: a review of context and utility for obesity prevention

Crane

Bohn‐Goldbaum

Grunseit

et al. 2020

Health Res Policy Sys

View full text Add to dashboard Cite

Background: Natural experiments are increasingly valued as a way to assess the health impact of health and nonhealth interventions when planned controlled experimental research designs may be infeasible or inappropriate to implement. This study sought to investigate the value of natural experiments by exploring how they have been used in practice. The study focused on obesity prevention research as one complex programme area for applying natural experiment studies. Methods: A literature search sought obesity prevention research from January 1997 to December 2017 and identified 46 population health studies that self-described as a natural experiment. Results: The majority of studies identified were published in the last 5 years, illustrating a more recent adoption of such opportunities. The majority of studies were evaluations of the impact of policies (n = 19), such as assessing changes to food labelling, food advertising or taxation on diet and obesity outcomes, or were built environment interventions (n = 17), such as the impact of built infrastructure on physical activity or access to healthy food. Research designs included quasi-experimental, pre-experimental and non-experimental methods. Few studies applied rigorous research designs to establish stronger causal inference, such as multiple pre/post measures, time series designs or comparison of change against an unexposed group. In general, researchers employed techniques to enhance the study utility but often were limited in the use of more rigorous study designs by ethical considerations and/or the particular context of the intervention. Conclusion: Greater recognition of the utility and versatility of natural experiments in generating evidence for complex health issues like obesity prevention is needed. This review suggests that natural experiments may be underutilised as an approach for providing evidence of the effects of interventions, particularly for evaluating health outcomes of interventions when unexpected opportunities to gather evidence arise.

show abstract

Integrating quasi-experimental and inductive designs in evaluation: A case study of the impact of free bus travel on public health

Cited by 25 publications

References 30 publications

Semiautomated text analytics for qualitative data synthesis

Semiautomated text analytics for qualitative data synthesis

Case study research for better evaluations of complex interventions: rationale and challenges

Using natural experiments to improve public health evidence: a review of context and utility for obesity prevention

Contact Info

Product

Resources

About