Combination of conditional random field with a rule based method in the extraction of PICO elements

Abstract

BackgroundExtracting primary care information in terms of Patient/Problem, Intervention, Comparison and Outcome, known as PICO elements, is difficult as the volume of medical information expands and the health semantics is complex to capture it from unstructured information. The combination of the machine learning methods (MLMs) with rule based methods (RBMs) could facilitate and improve the PICO extraction. This paper studies the PICO elements extraction methods. The goal is to combine the MLMs with the RBMs to extract PICO elements in medical papers to facilitate answering clinical questions formulated with the PICO framework.MethodsFirst, we analyze the aspects of the MLM model that influence the quality of the PICO elements extraction. Secondly, we combine the MLM approach with the RBMs in order to improve the PICO elements retrieval process. To conduct our experiments, we use a corpus of 1000 abstracts.ResultsWe obtain an F-score of 80% for P element, 64% for the I element and 92% for the O element. Given the nature of the used training corpus where P and I elements represent respectively only 6.5 and 5.8% of total sentences, the results are competitive with previously published ones.ConclusionsOur study of the PICO element extraction shows that the task is very challenging. The MLMs tend to have an acceptable precision rate but they have a low recall rate when the corpus is not representative. The RBMs backed up the MLMs to increase the recall rate and consequently the combination of the two methods gave better results.

Data extraction methods for systematic review (semi)automation: A living systematic review

“…Eight publications (15%) use rule-bases alone, while the rest of the publications use them in combination with other classifiers (data shown in Underlying data: Appendix A 86 ). Although used more frequently in the past, the five publications published between 2017 and now that use this approach combine it with conditional random fields (CRF), 24 use it alone, 25,26 use it with SVM 27 or use it with other binary classifiers. 28 In practice, these systems use rule-bases in the form of hand-crafted lists to identify candidate phrases for amount entities such as sample size 25,28 or to refine a result obtained by a machine-learning classifier on the entity level (e.g., instances where a specific intervention or outcome is extracted from a sentence).…”

“…28 In practice, these systems use rule-bases in the form of hand-crafted lists to identify candidate phrases for amount entities such as sample size 25,28 or to refine a result obtained by a machine-learning classifier on the entity level (e.g., instances where a specific intervention or outcome is extracted from a sentence). 24 Embedding and neural architectures are increasingly being used in literature from the past five years. In the 'Other' category, tools mentioned were mostly other classifiers such as maximum entropy classifiers (n = 3), kLog, J48, and various position or document-length classification algorithms.…”

“…The hyperparameter configuration or rule-base used to conceive a system may not retrieve comparable results in a different medical domain. 24,38 Therefore, scores might not be similar between different datasets, especially for rule-based classifiers, 54 when datasets are small, 26 when structure and distribution of class of interest varies, 24 or when the annotation guidelines vary. 58 Another caveat mentioned by 58,34 is that the size of the label space must be considered when comparing scores, as models that normalise to specific concepts rather than detecting entities tend to have lower precision, recall, and F1 scores.…”

Efficacy of Auricular Acupressure in Prevention and Treatment of Chemotherapy-Induced Nausea and Vomiting in Patients with Cancer: A Systematic Review and Meta-Analysis

“…PICO framework was used to form clinical queries and facilitate the literature search [ 36 ] ( Table 1 ). Relevant keywords were identified through a primary search on PubMed and Chinese Biological Medicine (CBM) to build the search strategies.…”

Search Characteristics and Reporting of Systematic Reviews of Animal Experiments: A Cross-Sectional Study

“…During the search procedure, sensitivity (recall) and accuracy (positive predictive value) are often used to evaluate the search results [41] . A number of researchers have found that the sensitivity (recall) of the P and I in the PICOS-based element search in SR/MA abstracts of clinical trials were 37-84% and 26-80%, respectively [42][43] . In addition, it has been shown that the sensitivity (recall) of P, I, and O were 62%, 47%, and 75%, respectively [47] when using an entity recognition model tool to analyze 191 biomedical articles.…”