Iterative guided machine learning-assisted systematic literature reviews: a diabetes case study

Abstract

Background Systematic Reviews (SR), studies of studies, use a formal process to evaluate the quality of scientific literature and determine ensuing effectiveness from qualifying articles to establish consensus findings around a hypothesis. Their value is increasing as the conduct and publication of research and evaluation has expanded and the process of identifying key insights becomes more time consuming. Text analytics and machine learning (ML) techniques may help overcome this problem of scale while still maintaining the level of rigor expected of SRs. Methods In this article, we discuss an approach that uses existing examples of SRs to build and test a method for assisting the SR title and abstract pre-screening by reducing the initial pool of potential articles down to articles that meet inclusion criteria. Our approach differs from previous approaches to using ML as a SR tool in that it incorporates ML configurations guided by previously conducted SRs, and human confirmation on ML predictions of relevant articles during multiple iterative reviews on smaller tranches of citations. We applied the tailored method to a new SR review effort to validate performance. Results The case study test of the approach proved a sensitivity (recall) in finding relevant articles during down selection that may rival many traditional processes and show ability to overcome most type II errors. The study achieved a sensitivity of 99.5% (213 out of 214) of total relevant articles while only conducting a human review of 31% of total articles available for review. Conclusions We believe this iterative method can help overcome bias in initial ML model training by having humans reinforce ML models with new and relevant information, and is an applied step towards transfer learning for ML in SR.

A Whole School, Whole Community, Whole Child Approach to Support Student Physical Activity and Nutrition: Introduction/Methods

“…Searches addressing specific PA and nutrition interventions (PA/nutrition in Figure 3) identified 8659 unique abstracts. To reduce the time burden of screening abstracts, we adopted an iterative machine learning (ML) approach, 46 [open] Recency of the evidence base is an important consideration; this information can be used to determine the scope of any necessary updates to an anchor review article.…”

A Rapid Semi-automated Literature Review on Legal Precedents Retrieval

“…Conducting an SLR in line with good practices and the necessary rigor level is a complex, multi-stage, and time-consuming task [15]. Text analytics and ML techniques may help overcome this scaling problem while still maintaining the level of rigor expected of SRs [16], which is the reason why literature review has been employed in many different fields such as banking [17], tourism and hospitality [18], and marketing [19].…”

Modelado de tópicos aplicado al análisis del papel del aprendizaje automático en revisiones sistemáticas

“…El tópico más relevante es el número 1, al estar conformado por el 50% de los documentos del corpus. Los documentos agrupados en este conjunto están relacionados con las tareas de clasificación de documentos, particularmente resúmenes y reportes cortos, en su mayoría relacionados con el campo de las ciencias médicas (Genc et al, 2020;Weißer et al, 2020;Zimmerman et al, 2021). En este tópico se destacan trabajos como el de Cohen et al (2009), quienes determinan cómo los métodos automatizados de RS se pueden mejorar utilizando datos de entrenamiento de los modelos de otras revisiones.…”