Innovation and forward‐thinking are needed to improve traditional synthesis methods: A response to Pescott and Stewart

Christie, Alec P.; Amano, Tatsuya; Martin, Philip A.; Shackelford, Gorm E.; Simmons, Benno I.; Sutherland, William J.

doi:10.1111/1365-2664.14154

Cited by 3 publications

(2 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…McCabe et al 68 produced an interactive web application to help psychology researchers visualise interaction effects, and communicate the statistical integrity of analyses: https://connorjmccabe.shinyapps.io/interactive/). For meta-analysis, 'dynamic meta-analysis' software has been developed, whereby effect sizes can be filtered and weighted, and results can be recalculated, using subgroup analysis, meta-regression, and recalibration 69 , which could be extended to alternate weighting schemes that incorporate generality criteria 37,70 . EviAtlas is an example of open source software for producing interactive visualisations of systematic map databases 71 .…”

Section: Action 2: Move Beyond Static Representations Of Ecological R...mentioning

confidence: 99%

Improving quantitative synthesis to achieve generality in ecology

et al. 2022

View full text Add to dashboard Cite

The inherent complexity and multi-causality of nature makes attempts to gain understanding and guidance about critical issues including rates of biodiversity loss, or the effectiveness of actions such as tree planting, wickedly difficult and often subject to poorly-substantiated assertions about their generality. Synthesis of primary ecological data is often assumed to achieve a notion of 'generality', through the quantification of overall effect sizes and consistency among studies, and has become a dominant research approach in ecology. Assertions about generality should raise the question: what exactly is generality? Ecologists rarely define either the generality of their findings, their estimand (what is estimated, based on the question of interest) or population of interest. Given that generality is fundamental to the philosophy of science, and the urgent need for scientific understanding and applied conservation actions to curb ecological breakdown, the loose use of the term 'generality' is problematic. In other disciplines generality is defined as comprising both generalisability: extending an inference about an estimand from the sample to the population, and transferability: the validity of estimand predictions in a different sampling unit or population. We review current practice in ecological synthesis, and demonstrate that by failing to define the assumptions that underpin generalisations and transfers of effect sizes, generality often misses its target. We then provide guidance for communicating nuanced inferences, and maximising the impact of syntheses both within and beyond academia. We finally propose pathways to generality applicable to ecological syntheses, which includes development of quantitative and qualitative criteria with which to license the transfer of estimands from both primary and synthetic studies.

show abstract

Section: Action 2: Move Beyond Static Representations Of Ecological R...mentioning

confidence: 99%

Improving quantitative synthesis to achieve generality in ecology

et al. 2022

View full text Add to dashboard Cite

show abstract

“…This has led to a growing scientific evidence base on conservation solutions (e.g., Conservation Evidence [http://www.conservationevidence.com; Sutherland et al, 2019], Evidensia [https://www.evidensia.eco/], Collaboration for Environmental Evidence [https://environmentalevidence.org/]), an increasing availability of global information on biodiversity status and threats (e.g., IUCN Red List; Stephenson & Stengel, 2020), and identification of important knowledge gaps (Christie et al, 2020; Christie, Amano, et al, 2021). Approaches toward gathering, synthesizing, and assessing evidence to inform conservation efforts are often focused on identifying general patterns, recommendations, and answers (Christie et al, 2020; Christie et al, 2022) across many scientific studies in a collated evidence base through meta‐analyses, various types of reviews, or subject‐wide evidence synthesis (Cook et al, 2017; Sutherland et al, 2004).…”

Section: Introductionmentioning

confidence: 99%

Assessing diverse evidence to improve conservation decision‐making

Christie,

Morgan,

Salafsky

et al. 2023

Conservat Sci and Prac

Self Cite

View full text Add to dashboard Cite

Meeting the urgent need to protect and restore ecosystems requires effective decision‐making through wisely considering a range of evidence. However, weighing and assessing evidence to make complex decisions is challenging, particularly when evidence is of diverse types, subjects, and sources, and varies greatly in its quality and relevance. To tackle these challenges, we present the Balance Evidence Assessment Method (BEAM), an intuitive way to weigh and assess the evidence relating to the core assumptions underpinning the planning and implementation of conservation projects, strategies, and actions. Our method directly tackles the question of how to bring together diverse evidence whilst assessing its relevance, reliability, and strength of support for a given assumption, which can be mapped, for example to a Theory of Change. We consider how simple principles and safeguards in applying this method could help to respectfully, and equitably, include more local forms of knowledge when assessing assumptions, such as by ensuring diverse groups of individuals contribute and assess evidence. The method can be flexibly applied within existing decision‐making tools, platforms, and frameworks whenever assumptions (i.e., claims and hypotheses) are made. This method could greatly facilitate and improve the weighing of diverse evidence to make decisions in a range of situations, from local projects to global policy platforms.

show abstract

Improving quantitative synthesis to achieve generality in ecology

Spake¹,

Nakagawa²,

Doncaster³

et al. 2022

Preprint

View full text Add to dashboard Cite

The inherent complexity and multi-causality of nature makes attempts to gain understanding and guidance about critical issues including rates of biodiversity loss, or the effectiveness of actions such as tree planting, wickedly difficult and often subject to poorly-substantiated assertions about their generality. Synthesis of primary ecological data is often assumed to achieve a notion of ‘generality’, through the quantification of overall effect sizes and consistency among studies, and has become a dominant research approach in ecology. Assertions about generality should raise the question: what exactly is generality? Ecologists rarely define either the generality of their findings, their estimand (what is estimated, based on the question of interest) or population of interest. Given that generality is fundamental to the philosophy of science, and the urgent need for scientific understanding and applied conservation actions to curb ecological breakdown, the loose use of the term ‘generality’ is problematic. In other disciplines generality is defined as comprising both generalisability: extending an inference about an estimand from the sample to the population, and transferability: the validity of estimand predictions in a different sampling unit or population. We review current practice in ecological synthesis, and demonstrate that by failing to define the assumptions that underpin generalisations and transfers of effect sizes, generality often misses its target. We then provide guidance for communicating nuanced inferences, and maximising the impact of syntheses both within and beyond academia. We finally propose pathways to generality applicable to ecological syntheses, which includes development of quantitative and qualitative criteria with which to license the transfer of estimands from both primary and synthetic studies.

show abstract

Innovation and forward‐thinking are needed to improve traditional synthesis methods: A response to Pescott and Stewart

Cited by 3 publications

References 72 publications

Improving quantitative synthesis to achieve generality in ecology

Improving quantitative synthesis to achieve generality in ecology

Assessing diverse evidence to improve conservation decision‐making

Improving quantitative synthesis to achieve generality in ecology

Contact Info

Product

Resources

About