Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses

Hamel, Candyce; Hersi, Mona; Kelly, Shannon; Tricco, Andrea C.; Straus, Sharon E.; Wells, George A.; Pham, Ba’; Hutton, Brian

doi:10.1186/s12874-021-01451-2

Cited by 41 publications

(39 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Additionally, the results show that authors should involve translation software or translators in the conduct of rapid and systematic reviews including non-English articles, either internally or through a research network. Generally, the growing use of automation software may prevent the false exclusion of studies [ 19 ]. This would require providing a clean, high-quality initial data set to train the algorithm, ensuring that any duplicates with conflicting decisions are removed and prespecifying records correctly as includes or excludes [ 19 ].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Characteristics and recovery methods of studies falsely excluded during literature screening—a systematic review

et al. 2022

View full text Add to dashboard Cite

Background Due to the growing need to provide evidence syntheses under time constraints, researchers have begun focusing on the exploration of rapid review methods, which often employ single-reviewer literature screening. However, single-reviewer screening misses, on average, 13% of relevant studies, compared to 3% with dual-reviewer screening. Little guidance exists regarding methods to recover studies falsely excluded during literature screening. Likewise, it is unclear whether specific study characteristics can predict an increased risk of false exclusion. This systematic review aimed to identify supplementary search methods that can be used to recover studies falsely excluded during literature screening. Moreover, it strove to identify study-level predictors that indicate an elevated risk of false exclusions of studies during literature screening. Methods We performed literature searches for eligible studies in MEDLINE, Science Citation Index Expanded, Social Sciences Citation Index, Current Contents Connect, Embase, Epistemonikos.org, and Information Science & Technology Abstracts from 1999 to June 23, 2020. We searched for gray literature, checked reference lists, and conducted hand searches in two relevant journals and similar article searches current to January 28, 2021. Two investigators independently screened the literature; one investigator performed the data extraction, and a second investigator checked for correctness and completeness. Two reviewers assessed the risk of bias of eligible studies. We synthesized the results narratively. Results Three method studies, two with a case-study design and one with a case-series design, met the inclusion criteria. One study reported that all falsely excluded publications (8%) could be recovered through reference list checking compared to other supplementary search methods. No included methods study analyzed the impact of recovered studies on conclusions or meta-analyses. Two studies reported that up to 8% of studies were falsely excluded due to uninformative titles and abstracts, and one study showed that 11% of non-English studies were falsely excluded. Conclusions Due to the limited evidence based on two case studies and one case series, we can draw no firm conclusion about the most reliable and most valid method to recover studies falsely excluded during literature screening or about the characteristics that might predict a higher risk of false exclusion. Systematic review registration https://osf.io/v2pjr/

show abstract

Section: Discussionmentioning

confidence: 99%

“…Generally, the growing use of automation software may prevent the false exclusion of studies [ 19 ]. This would require providing a clean, high-quality initial data set to train the algorithm, ensuring that any duplicates with conflicting decisions are removed and prespecifying records correctly as includes or excludes [ 19 ].…”

Section: Discussionmentioning

confidence: 99%

Characteristics and recovery methods of studies falsely excluded during literature screening—a systematic review

et al. 2022

View full text Add to dashboard Cite

show abstract

“… 64 For stage 1, we will use Distiller SR’s artificial intelligence (AI) active-machine learning feature to prioritise title and abstract screening of citations. 65 66 This method has been validated. 66 This active-machine learning feature will allow us to perform prioritised screening, as a relevance score will be generated for each citation during an initial training exercise on a sample of approximately 200 citations; this feature will continue to learn throughout the stage 1 screening process, presenting reviewers with the most relevant citations first.…”

Section: Methods and Analysismentioning

confidence: 99%

Intraoperative pharmacologic opioid minimisation strategies and patient-centred outcomes after surgery: a scoping review protocol

et al. 2023

Self Cite

View full text Add to dashboard Cite

IntroductionFor close to a century opioid administration has been a standard of care to complement anaesthesia during surgery. Considering the worldwide opioid epidemic, this practice is now being challenged and there is a growing use of systemic pharmacological opioid minimising strategies. Our aim is to conduct a scoping review that will examine clinical trials that have evaluated the impact of intraoperative opioid minimisation strategies on patient-centred outcomes and identify promising strategies.Methods and analysisOur scoping review will follow the framework developed by Arksey and O’Malley. We will search MEDLINE, Embase, CENTRAL, Web of Science and CINAHL from their inception approximately in March 2023. We will include randomised controlled trials, assessing the impact of systemic intraoperative pharmacologic opioid minimisation strategies on patient-centred outcomes. We define an opioid minimisation strategy as any non-opioid drug with antinociceptive properties administered during the intraoperative period. Patient-centred outcomes will be defined and classified based on the consensus definitions established by the Standardised Endpoints in Perioperative Medicine initiative (StEP-COMPAC group) and informed by knowledge users and patient partners. We will use a coproduction approach involving interested parties. Our multidisciplinary team includes knowledge users, patient partners, methodologists and knowledge user organisations. Knowledge users will provide input on methods, outcomes, clinical significance of findings, implementation and feasibility. Patient partners will participate in assessing the relevance of our design, methods and outcomes and help to facilitate evidence translation. We will provide a thorough description of available clinical trials, compare their reported patient-centred outcome measures with established recommendations and identify promising strategies.Ethics and disseminationEthics approval is not required for the review. Our scoping review will inform future research including clinical trials and systematic reviews through identification of important intraoperative interventions. Results will be disseminated through a peer-reviewed publication, presentation at conferences and through our network of knowledge user collaborators.RegistrationOpen Science Foundation (currently embargoed)

show abstract

“…A human reviewer will screen all citations excluded by the AI reviewer, with any conflicts to be resolved by two human reviewers. Our approach is in alignment with recent guidance for the use of AI in knowledge syntheses 46. All potentially relevant full-text articles will be screened by independent reviewers in duplicate for eligibility, with conflicts to be resolved through discussion or consultation with a third reviewer until consensus is reached.…”

Section: Methods and Analysismentioning

confidence: 99%

Service-level barriers to and facilitators of access to services for the treatment of alcohol use disorder and problematic alcohol use: protocol for a scoping review

et al. 2022

Self Cite

View full text Add to dashboard Cite

IntroductionPrior to the COVID-19 pandemic, substance use health services for treatment of alcohol use disorder and problematic alcohol use (AUD/PAU) were fragmented and challenging to access. The pandemic magnified system weaknesses, often resulting in disruptions of treatment as alcohol use during the pandemic rose. When treatment services were available, utilisation was often low for various reasons. Virtual care was implemented to offset the drop in in-person care, however accessibility was not universal. Identification of the characteristics of treatment services for AUD/PAU that impact accessibility, as perceived by the individuals accessing or providing the services, will provide insights to enable improved access. We will perform a scoping review that will identify characteristics of services for treatment of AUD/PAU that have been identified as barriers to or facilitators of service access from the perspectives of these groups.Methods and analysisWe will follow scoping review methodological guidance from the Joanna Briggs Institute. Using the OVID platform, we will search Ovid MEDLINE including Epub Ahead of Print and In-Process and Other Non-Indexed Citations, Embase Classic+Embase, APA PsychInfo, Cochrane Register of Controlled Trials, the Cochrane Database of Systematic Reviews and CINAHL (Ebsco Platform). Multiple reviewers will screen citations. We will seek studies reporting data collected from individuals with AUD/PAU or providers of treatment for AUD/PAU on service-level factors affecting access to care. We will map barriers to and facilitators of access to AUD/PAU treatment services identified in the relevant studies, stratified by service type and key measures of inequity across service users.Ethics and disseminationThis research will enhance awareness of existing evidence regarding barriers to and facilitators of access to services for the treatment of alcohol use disorder and problematic alcohol use. Findings will be disseminated through publications, conference presentations and a stakeholder meeting. As this is a scoping review of published literature, no ethics approval was required.

show abstract

Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses

Cited by 41 publications

References 40 publications

Characteristics and recovery methods of studies falsely excluded during literature screening—a systematic review

Characteristics and recovery methods of studies falsely excluded during literature screening—a systematic review

Intraoperative pharmacologic opioid minimisation strategies and patient-centred outcomes after surgery: a scoping review protocol

Service-level barriers to and facilitators of access to services for the treatment of alcohol use disorder and problematic alcohol use: protocol for a scoping review

Contact Info

Product

Resources

About