EviAtlas: a tool for visualising evidence synthesis databases

Haddaway, Neal R.; Feierman, Andrew; Grainger, Matthew; Gray, Charles T.; Tanriver-Ayder, Ezgi; Dhaubanjar, Sanita; Westgate, Martin J.

doi:10.1186/s13750-019-0167-1

Cited by 52 publications

(29 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Authors could therefore adopt some 'mapping' tools (e.g. Haddaway et al, 2019) to distil the key characteristics of their data set in a concise (and sometimes beautiful) format and summarise the magnitude and direction of research biases (Item 16).…”

Section: Item 19: Results Of Study Selection Processmentioning

confidence: 99%

Preferred reporting items for systematic reviews and meta‐analyses in ecology and evolutionary biology: a PRISMA extension

et al. 2021

View full text Add to dashboard Cite

Since the early 1990s, ecologists and evolutionary biologists have aggregated primary research using meta-analytic methods to understand ecological and evolutionary phenomena. Meta-analyses can resolve long-standing disputes, dispel spurious claims, and generate new research questions. At their worst, however, meta-analysis publications are wolves in sheep's clothing: subjective with biased conclusions, hidden under coats of objective authority. Conclusions can be rendered unreliable by inappropriate statistical methods, problems with the methods used to select primary research, or problems within the primary research itself. Because of these risks, meta-analyses are increasingly conducted as part of systematic reviews, which use structured, transparent, and reproducible methods to collate and summarise evidence. For readers to determine whether the conclusions from a systematic review or meta-analysis should be trustedand to be able to build upon the reviewauthors need to report what they did, why they did it, and what they found. Complete, transparent, and reproducible reporting is measured by 'reporting quality'. To assess perceptions and standards of reporting quality of systematic reviews and meta-analyses published in ecology and evolutionary biology, we surveyed 208 researchers with relevant experience (as authors, reviewers, or editors), and conducted detailed evaluations of 102 systematic review and meta-analysis papers published between 2010 and 2019. Reporting quality was far below optimal and approximately normally distributed. Measured reporting quality was lower than what the community perceived, particularly for the systematic review methods required to measure trustworthiness. The minority of assessed papers that referenced a guideline (16%) showed substantially higher reporting quality than average, and surveyed researchers showed interest in using a reporting guideline to improve reporting quality. The leading guideline for improving reporting quality of systematic reviews is the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement. Here we unveil an extension of PRISMA to serve the meta-analysis community in ecology and evolutionary biology: PRISMA-EcoEvo (version 1.0). PRISMA-EcoEvo is a checklist of 27 main items that, when applicable, should be reported in systematic review and meta-analysis publications summarising primary research in ecology and evolutionary biology. In this explanation and elaboration document, we provide guidance for authors, reviewers, and editors, with explanations for each item on the checklist, including supplementary examples from published papers. Authors can

show abstract

Section: Item 19: Results Of Study Selection Processmentioning

confidence: 99%

Preferred reporting items for systematic reviews and meta‐analyses in ecology and evolutionary biology: a PRISMA extension

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Indeed, by embracing standard approaches to database design, we believe that the job of producing and using databases can be made far easier through automation tools (e.g. using EviAtlas; [40]). We eagerly anticipate future toolchain walkthroughs that provide subdiscipline-or tool-specific code for structuring data for evidence synthesis.…”

Section: Discussionmentioning

confidence: 99%

Novel tools and methods for designing and wrangling multifunctional, machine-readable evidence synthesis databases

2021

Self Cite

View full text Add to dashboard Cite

One of the most important steps in the process of conducting a systematic review or map is data extraction and the production of a database of coding, metadata and study data. There are many ways to structure these data, but to date, no guidelines or standards have been produced for the evidence synthesis community to support their production. Furthermore, there is little adoption of easily machine-readable, readily reusable and adaptable databases: these databases would be easier to translate into different formats by review authors, for example for tabulation, visualisation and analysis, and also by readers of the review/map. As a result, it is common for systematic review and map authors to produce bespoke, complex data structures that, although typically provided digitally, require considerable efforts to understand, verify and reuse. Here, we report on an analysis of systematic reviews and maps published by the Collaboration for Environmental Evidence, and discuss major issues that hamper machine readability and data reuse or verification. We highlight different justifications for the alternative data formats found: condensed databases; long databases; and wide databases. We describe these challenges in the context of data science principles that can support curation and publication of machine-readable, Open Data. We then go on to make recommendations to review and map authors on how to plan and structure their data, and we provide a suite of novel R-based functions to support efficient and reliable translation of databases between formats that are useful for presentation (condensed, human readable tables), filtering and visualisation (wide databases), and analysis (long databases). We hope that our recommendations for adoption of standard practices in database formatting, and the tools necessary to rapidly move between formats will provide a step-change in transparency and replicability of Open Data in evidence synthesis.

show abstract

“…Instead, the features provided by revtools are likely to be particularly powerful when used in combination with those of related packages. Currently, for example, it is possible to plan a search strategy with litsearchr 35 ; screen your articles with revtools; download articles using fulltext 36 ; analyze the resulting data in metafor 37 or one of the many other meta-analysis packages available in R 9 ; and present a database of study results using EviAtlas 38 . While this is encouraging, many stages of the systematic review workflow remain unsupported within R, with data extraction being a key gap.…”

Section: Discussionmentioning

confidence: 99%

revtools: An R package to support article screening for evidence synthesis

Westgate

2019

Research Synthesis Methods

Self Cite

167

View full text Add to dashboard Cite

The field of evidence synthesis is growing rapidly, with a corresponding increase in the number of software tools and workflows to support the construction of systematic reviews, systematic maps, and meta-analyses. Despite much progress, however, a number of problems remain including slow integration of new statistical or methodological approaches into userfriendly software, low prevalence of open-source software, and poor integration among distinct software tools. These issues hinder the utility and transparency of new methods to the research community. Here I present revtools, an R package to support article screening during evidence synthesis projects. It provides tools for the import and de-duplication of bibliographic data, screening of articles by title or abstract, and visualization of article content using topic models. The software is entirely open-source and combines commandline scripting for experienced programmers with custom-built user interfaces for casual users, with further methods to support article screening to be added over time. Revtools provides free access to novel methods in an open-source environment, and represents a valuable step in expanding the capacity of R to support evidence synthesis projects. .

show abstract

EviAtlas: a tool for visualising evidence synthesis databases

Cited by 52 publications

References 21 publications

Preferred reporting items for systematic reviews and meta‐analyses in ecology and evolutionary biology: a PRISMA extension

Preferred reporting items for systematic reviews and meta‐analyses in ecology and evolutionary biology: a PRISMA extension

Novel tools and methods for designing and wrangling multifunctional, machine-readable evidence synthesis databases

revtools: An R package to support article screening for evidence synthesis

Contact Info

Product

Resources

About