Improving reference prioritisation with PICO recognition

Abstract

BackgroundMachine learning can assist with multiple tasks during systematic reviews to facilitate the rapid retrieval of relevant references during screening and to identify and extract information relevant to the study characteristics, which include the PICO elements of patient/population, intervention, comparator, and outcomes. The latter requires techniques for identifying and categorising fragments of text, known as named entity recognition.MethodsA publicly available corpus of PICO annotations on biomedical abstracts is used to train a named entity recognition model, which is implemented as a recurrent neural network. This model is then applied to a separate collection of abstracts for references from systematic reviews within biomedical and health domains. The occurrences of words tagged in the context of specific PICO contexts are used as additional features for a relevancy classification model. Simulations of the machine learning-assisted screening are used to evaluate the work saved by the relevancy model with and without the PICO features. Chi-squared and statistical significance of positive predicted values are used to identify words that are more indicative of relevancy within PICO contexts.ResultsInclusion of PICO features improves the performance metric on 15 of the 20 collections, with substantial gains on certain systematic reviews. Examples of words whose PICO context are more precise can explain this increase.ConclusionsWords within PICO tagged segments in abstracts are predictive features for determining inclusion. Combining PICO annotation model into the relevancy classification pipeline is a promising approach. The annotations may be useful on their own to aid users in pinpointing necessary information for data extraction, or to facilitate semantic search.

Subcutaneous vaccine administration – an outmoded practice

“… US adults 18–34 y old. n = 72 Live attenuated Vaccinia vaccine; Modified Vaccine Ankara (MVA) 10 7 or 10 8 TCID 50 Data for n = 40 IM n = 20 SC n = 20 Reactogenicity SC > IM, p < .05 for severe local erythema and induration (31–70 mm) Immunogenicity IM and SC comparable antibody responses. Vollmar et al 52 Single-center, randomized, double-blind, phase I study.…”

“…The purpose of this review is to use EBM to seek to rationalize the route of administration of vaccines given by SC, IM or either routes. The PICO elements 8 for this review are P = human vaccine recipients, I = intramuscular route of injection, C = subcutaneous route of injection and O = reactogenicity and immunogenicity of vaccines.…”

Toward Assessing Clinical Trial Publications for Reporting Transparency

“…Much attention has been paid to PICO elements (Population, Intervention, Comparator, and Outcome), used in evidence-based medicine [16] to capture the most salient aspects of clinical intervention studies [27–30]. PICO elements were annotated at the noun phrase level [27, 29] as well as the sentence level [27, 28].…”

“…In their corpus, some PICO elements are further subcategorized at more granular levels (e.g., Physical Health as a subcategory of Outcomes). Text mining methods used to extract PICO characteristics ran the gamut from early knowledge-based and traditional machine learning methods [27] to more recent semi-supervised [28] and neural network models, including Long Short-Term Memory (LSTM) and recurrent neural networks (RNN) [29, 30]. PICO variants, such as PIBOSO (B: background, S: study design, O: other), have also been considered.…”

Seed-driven document ranking for systematic reviews in evidence-based medicine

“…Referring to Figure 4.2, the positional patterns of PICO elements can be seen in a collective manner among several thousands documents and the pattern in a pair of two documents may not be strong enough. Besides, using positional information is a soft way of incorporating PICO elements in matching as the PICO spans are not explicitly pre-detected [86].…”