Natural language processing was effective in assisting rapid title and abstract screening when updating systematic reviews

Abstract

Background and Objective: To examine whether the use of natural language processing (NLP) technology is effective in assisting rapid title and abstract screening when updating a systematic review.Study Design: Using the searched literature from a published systematic review, we trained and tested an NLP model that enables rapid title and abstract screening when updating a systematic review. The model was a light gradient boosting machine (LightGBM), an ensemble learning classifier which integrates four pretrained Bidirectional Encoder Representations from Transformers (BERT) models. We divided the searched citations into two sets (ie, training and test sets). The model was trained using the training set and assessed for screening performance using the test set. The searched citations, whose eligibility was determined by two independent reviewers, were treated as the reference standard.Results: The test set included 947 citations; our model included 340 citations, excluded 607 citations, and achieved 96% sensitivity, and 78% specificity. If the classifier assessment in the case study was accepted, reviewers would lose 8 of 180 eligible citations (4%), none of which were ultimately included in the systematic review after full-text consideration, while decreasing the workload by 64.1%. Conclusion:NLP technology using the ensemble learning method may effectively assist in rapid literature screening when updating systematic reviews.

Methodological guidance for rapid reviews in healthcare: A scoping review

“…18,49 Software engineering methods such as Natural Language Processing have been used to screen articles, analyze their content, chart results, and so forth. 65,66 For example, a substantial reduction in screening burden with corresponding time savings could be shown by using an artificial intelligence supported software tool. 67 Although automation may be particularly interesting for the time-critical production of RRs, there is little evidence on the validity of these techniques.…”

“…In two other papers, automation techniques are mentioned in the discussion sections, but are not part of the recommendation 18,49 . Software engineering methods such as Natural Language Processing have been used to screen articles, analyze their content, chart results, and so forth 65,66 . For example, a substantial reduction in screening burden with corresponding time savings could be shown by using an artificial intelligence supported software tool 67 .…”

Toward better translation of clinical research evidence into rapid recommendations for traditional Chinese medicine interventions: A methodological framework

“…Compared with the established rapid recommendation framework, the unique features of MESERT approach are the inclusion of multi-dimensional evidence appropriate for TCM interventions, systematic synthesis of multi-dimensional evidence, assessment of the strength of causal effects in the synthesized evidence. Novel techniques, such as natural language processing, 12 could help evidence identification and screening and accelerate evidence translation.…”

Graph Model and Deep Learning for Topic Labels in Classifying Short Texts of Scientific Article Titles

“…As a result, our proposed framework offers procedures to process non-English texts for preparing a searching system in a cold-start situation. Different studies on a recommendation for systematic reviews had a natural language approach with the recent famed deep learning architecture of bidirectional encoder representations from transformers (BERT) [12]. The studies showed that the ensemble learning model increases sensitivity and specificity values.…”

“…Hence, the deep learning approach is convenient for classifying short texts [6][7][8][9]. However, the possibility of overlapped context needs additional information from word concepts of knowledge base [10] or predefined categories of classification system [11,12].…”