Causal assessment in evidence synthesis: A methodological review of reviews

Shimonovich, Michal; Pearce, Anna; Thomson, Hilary; Katikireddi, Srinivasa Vittal

doi:10.1002/jrsm.1569

Cited by 5 publications

(12 citation statements)

References 93 publications

(669 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We considered 'analogy' and 'coherence' viewpoints to have limited utility and were thus excluded. In our methodological review of 'causal' reviews in population health, 10 we found they are also the least used viewpoints.…”

Section: Overview Of Bradford Hill Viewpoints In Systematic Reviewsmentioning

confidence: 95%

“…33 Our approach to operationalising BH viewpoints may differ from how they are used in other 'causal' reviews. 10 We included viewpoints which remained relevant to recent developments in causal inference (described in Table 1). We previously found overlap between causal approaches grounded in counterfactual thinking 33 and the BH viewpoints experiment, strength of association, specificity, dose-response relationship, consistency, temporality, and plausibility.…”

Section: Overview Of Bradford Hill Viewpoints In Systematic Reviewsmentioning

confidence: 99%

“…3,9 There is wide-spread use of Bradford Hill (BH) viewpoints within systematic reviews of non-randomised studies to support causal assessment. [10][11][12][13] Going forward, we will refer to systematic reviews which focus on assessing causality and have explicitly incorporated tools, guidelines, or methods to support causal assessment as 'causal' systematic reviews.…”

Section: Introductionmentioning

confidence: 99%

“…16,17 Despite their prominence, there are currently no agreed upon guidelines for how BH viewpoints should be used. 10,18 Bradford Hill himself argued against imposing "hard-and-fast rules of evidence that must be observed before we accept cause and effect." 14, p. 299 Though many epidemiologists have revisited BH viewpoints to elucidate how they should be used, 12,[19][20][21] considerable disagreement remains over the theoretical underpinning of each BH viewpoint.…”

Section: Introductionmentioning

confidence: 99%

“…20,22 The lack of guidance and ongoing disagreements may explain the considerable variation in how BH viewpoints are used and reported in population health systematic reviews. 10 Transparently reporting how BH viewpoints are used in so-called 'causal' systematic reviews is important for scrutinising causal claims, including elucidating the reasons for inconsistent or contradictory conclusions. 6,18,22 Given the considerable variation in how BH viewpoints are used in systematic reviews, and the lack of transparency which makes it difficult to assess reproducibility, we sought to develop an approach that might overcome these limitations.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Applying Bradford Hill to assessing causality in systematic reviews: a transparent approach using process tracing

Shimonovich¹,

Thomson²,

Pearce³

et al. 2023

Preprint

View full text Add to dashboard Cite

Background: Bradford Hill (BH) viewpoints are widely used to assess causality in systematic reviews, but their application has often lacked reproducibility. We describe an approach for assessing causality within systematic reviews (‘causal’ reviews), illustrating its application to the topic of income inequality and health. Our approach draws on principles of process tracing, a method used for case study research, to harness BH viewpoints to judge evidence for causal claims. Methods: In process tracing, a hypothesis may be confirmed by observing highly unique evidence and disconfirmed by observing highly definitive evidence. We drew on these principles to consider the value of finding supportive or contradictory evidence for each BH viewpoint characterised by its uniqueness and definitiveness.Results: In our exemplar systematic review, we hypothesised that income inequality adversely affects self-rated health and all-cause mortality. BH viewpoints ‘analogy’ and ‘coherence’ were excluded from the causal assessment because of their low uniqueness and low definitiveness. The ‘experiment’ viewpoint was considered highly unique and highly definitive, and thus could be particularly valuable. We propose five steps for using BH viewpoints in a ‘causal’ review: 1) define the hypothesis; 2) characterise each viewpoint; 3) specify the evidence expected for each BH viewpoint for a true or untrue hypothesis; 4) gather evidence for each viewpoint (e.g., systematic review meta-analyses, critical appraisal, background knowledge); 5) consider if each viewpoint was met (supportive evidence) or unmet (contradictory evidence).Conclusions: Incorporating process tracing has the potential to provide transparency and structure when using BH viewpoints in ‘causal’ reviews.

show abstract

Section: Overview Of Bradford Hill Viewpoints In Systematic Reviewsmentioning

confidence: 95%

Section: Overview Of Bradford Hill Viewpoints In Systematic Reviewsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Applying Bradford Hill to assessing causality in systematic reviews: a transparent approach using process tracing

Shimonovich¹,

Thomson²,

Pearce³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Causal assessment in evidence synthesis: A methodological review of reviews

Shimonovich

Pearce

Thomson

et al. 2022

Research Synthesis Methods

View full text Add to dashboard Cite

In fields (such as population health) where randomised trials are often lacking, systematic reviews (SRs) can harness diversity in study design, settings and populations to assess the evidence for a putative causal relationship. SRs may incorporate causal assessment approaches (CAAs), sometimes called 'causal reviews', but there is currently no consensus on how these should be conducted. We conducted a methodological review of self-identifying 'causal reviews' within the field of population health to establish: (1) which CAAs are used; (2) differences in how CAAs are implemented; (3) how methods were modified to incorporate causal assessment in SRs. Three databases were searched and two independent reviewers selected reviews for inclusion. Data were extracted using a standardised form and summarised using tabulation and narratively. Fifty-three reviews incorporated CAAs: 46/53 applied Bradford Hill (BH) viewpoints/criteria, with the remainder taking alternative approaches: Medical Research Council guidance on natural experiments (2/53, 3.8%); realist reviews (2/53, 3.8%); horizontal SRs (1/53, 1.9%); 'sign test' of causal mechanisms (1/53, 1.9%); and a causal cascade model (1/53, 1.9%). Though most SRs incorporated BH, there was variation in application and transparency. There was considerable overlap across the CAAs, with a tradeoff between breadth (BH viewpoints considered a greater range of causal characteristics) and depth (many alternative CAAs focused on one viewpoint). Improved transparency in the implementation of CAA in SRs in needed to ensure their validity and allow robust assessments of causality within evidence synthesis.

show abstract

Causal Assessment of Income Inequality on Self‐Rated Health and All‐Cause Mortality: A Systematic Review and Meta‐Analysis

SHIMONOVICH,

CAMPBELL,

THOMSON

et al. 2024

Milbank Quarterly

Self Cite

View full text Add to dashboard Cite

Policy Points Income is thought to impact a broad range of health outcomes. However, whether income inequality (how unequal the distribution of income is in a population) has an additional impact on health is extensively debated. Studies that use multilevel data, which have recently increased in popularity, are necessary to separate the contextual effects of income inequality on health from the effects of individual income on health. Our systematic review found only small associations between income inequality and poor self‐rated health and all‐cause mortality. The available evidence does not suggest causality, although it remains methodologically flawed and limited, with very few studies using natural experimental approaches or examining income inequality at the national level. ContextWhether income inequality has a direct effect on health or is only associated because of the effect of individual income has long been debated. We aimed to understand the association between income inequality and self‐rated health (SRH) and all‐cause mortality (mortality) and assess if these relationships are likely to be causal.MethodsWe searched Medline, ISI Web of Science, Embase, and EconLit (PROSPERO: CRD42021252791) for studies considering income inequality and SRH or mortality using multilevel data and adjusting for individual‐level socioeconomic position. We calculated pooled odds ratios (ORs) for poor SRH and relative risk ratios (RRs) for mortality from random‐effects meta‐analyses. We critically appraised included studies using the Risk of Bias in Nonrandomized Studies – of Interventions tool. We assessed certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation framework and causality using Bradford Hill (BH) viewpoints.FindingsThe primary meta‐analyses included 2,916,576 participants in 38 cross‐sectional studies assessing SRH and 10,727,470 participants in 14 cohort studies of mortality. Per 0.05‐unit increase in the Gini coefficient, a measure of income inequality, the ORs and RRs (95% confidence intervals) for SRH and mortality were 1.06 (1.03‐1.08) and 1.02 (1.00‐1.04), respectively. A total of 63.2% of SRH and 50.0% of mortality studies were at serious risk of bias (RoB), resulting in very low and low certainty ratings, respectively. For SRH and mortality, we did not identify relevant evidence to assess the specificity or, for SRH only, the experiment BH viewpoints; evidence for strength of association and dose–response gradient was inconclusive because of the high RoB; we found evidence in support of temporality and plausibility.ConclusionsIncreased income inequality is only marginally associated with SRH and mortality, but the current evidence base is too methodologically limited to support a causal relationship. To address the gaps we identified, future research should focus on income inequality measured at the national level and addressing confounding with natural experiment approaches.

show abstract

Causal assessment in evidence synthesis: A methodological review of reviews

Cited by 5 publications

References 93 publications

Applying Bradford Hill to assessing causality in systematic reviews: a transparent approach using process tracing

Applying Bradford Hill to assessing causality in systematic reviews: a transparent approach using process tracing

Causal assessment in evidence synthesis: A methodological review of reviews

Causal Assessment of Income Inequality on Self‐Rated Health and All‐Cause Mortality: A Systematic Review and Meta‐Analysis

Contact Info

Product

Resources

About