Reviewing retrieved references for inclusion in systematic reviews using EndNote

“…Screening references is a lengthy process, and double screening with two reviewers is recommended to avoid missing references . Given that the estimated screening time per reference (title and abstract) is between 30 seconds and 1 minute and the estimated time to discuss and resolve an inclusion disagreement between reviewers is approximately 5 minutes per reference, screening 5000 references will last between 83 to 125 hours per reviewer . The estimated costs are also considerable: amounting to £13 000 for a single review …”

Section: Introductionmentioning

confidence: 99%

Prioritising references for systematic reviews with RobotAnalyst: A user study

Przybyła

Brockmeier

Kontonatsios

et al. 2018

Research Synthesis Methods

100

View full text Add to dashboard Cite

Screening references is a time‐consuming step necessary for systematic reviews and guideline development. Previous studies have shown that human effort can be reduced by using machine learning software to prioritise large reference collections such that most of the relevant references are identified before screening is completed. We describe and evaluate RobotAnalyst, a Web‐based software system that combines text‐mining and machine learning algorithms for organising references by their content and actively prioritising them based on a relevancy classification model trained and updated throughout the process. We report an evaluation over 22 reference collections (most are related to public health topics) screened using RobotAnalyst with a total of 43 610 abstract‐level decisions. The number of references that needed to be screened to identify 95% of the abstract‐level inclusions for the evidence review was reduced on 19 of the 22 collections. Significant gains over random sampling were achieved for all reviews conducted with active prioritisation, as compared with only two of five when prioritisation was not used. RobotAnalyst's descriptive clustering and topic modelling functionalities were also evaluated by public health analysts. Descriptive clustering provided more coherent organisation than topic modelling, and the content of the clusters was apparent to the users across a varying number of clusters. This is the first large‐scale study using technology‐assisted screening to perform new reviews, and the positive results provide empirical evidence that RobotAnalyst can accelerate the identification of relevant studies. The results also highlight the issue of user complacency and the need for a stopping criterion to realise the work savings.

show abstract

“…As a rule of thumb, information specialists and experienced reviewers find it feasible to screen between 100 and 150 abstracts within 1 h [55]; for inexperienced researchers, this would be a lower number. Clear documentation of both manually or electronically preformed searches is essential for the reproducibility of your study findings [56].…”

Section: Step 23: Perform Searchesmentioning

confidence: 99%

“…For model-based use EEs use the NICE checklist [11]. Both review types 3.3 Transferability assessment The Welte checklist [55] is recommended for all trial-based and model-based EEs.…”

Section: Risk Of Bias Assessment Both Review Typesmentioning

confidence: 99%

How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: a five-step approach (part 1/3)

Mastrigt

Hiligsmann

Arts

et al. 2016

Expert Review of Pharmacoeconomics & Outcomes Research

150

159

View full text Add to dashboard Cite

Introduction: Systematic reviews of economic evaluations are useful for synthesizing economic evidence about health interventions and for informing evidence-based decisions. Areas covered: As there is no detailed description of the methods for performing a systematic review of economic evidence, this paper aims to provide an overview of state-of-the-art methodology. This is laid out in a 5-step approach, as follows: step 1) initiating a systematic review; step 2) identifying (full) economic evaluations; step 3) data extraction, risk of bias and transferability assessment; step 4) reporting results; step 5) discussion and interpretation of findings. Expert commentary: The paper aims to help inexperienced reviewers and clinical practice guideline developers, but also to be a resource for experts in the field who want to check on current methodological developments. ARTICLE HISTORY

show abstract

Reviewing retrieved references for inclusion in systematic reviews using EndNote

Abstract: Not applicable

Cited by 158 publications

References 7 publications

Frequency of submicroscopic chromosomal aberrations in pregnancies without increased risk for structural chromosomal aberrations: systematic review and meta‐analysis

Frequency of submicroscopic chromosomal aberrations in pregnancies without increased risk for structural chromosomal aberrations: systematic review and meta‐analysis

Prioritising references for systematic reviews with RobotAnalyst: A user study

How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: a five-step approach (part 1/3)

Contact Info

Product

Resources

About