PICO extraction by combining the robustness of machine-learning methods with the rule-based methods

Chabou, Samir; Iglewski, Michal

doi:10.1109/wcitca.2015.7367038

Cited by 7 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… P, Condition, IC, 0 633 abstracts for O, 100 for other classes No 51 Entities Age, Design, Setting (Country), IC, N, study dates and affiliated institutions 185 full texts (at least 93 labelled) No 53 Sentences and entities P, IC, Age, Gender, Design, Condition, Race 2000 sentences from abstracts No 66 200 abstracts, 140 contain sentence and entity labels P, IC 200 abstracts No 80 Auto-labelled structured abstracts, sentence level. P, IC, O 14200+ abstracts No 67 Entities P, age, gender, race 50 abstracts No 81 Sentences (and entities?) P, IC, O 3000 abstracts No 28 Entities …”

Section: Resultsmentioning

confidence: 99%

Data extraction methods for systematic review (semi)automation: A living systematic review

et al. 2021

View full text Add to dashboard Cite

Background: The reliable and usable (semi)automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies. Methods: We systematically and continually search MEDLINE, Institute of Electrical and Electronics Engineers (IEEE), arXiv, and the dblp computer science bibliography databases. Full text screening and data extraction are conducted within an open-source living systematic review application created for the purpose of this review. This iteration of the living review includes publications up to a cut-off date of 22 April 2020. Results: In total, 53 publications are included in this version of our review. Of these, 41 (77%) of the publications addressed extraction of data from abstracts, while 14 (26%) used full texts. A total of 48 (90%) publications developed and evaluated classifiers that used randomised controlled trials as the main target texts. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. A description of their datasets was provided by 49 publications (94%), but only seven (13%) made the data publicly available. Code was made available by 10 (19%) publications, and five (9%) implemented publicly available tools. Conclusions: This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of systematic review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting epidemiological or diagnostic accuracy data. The lack of publicly available gold-standard data for evaluation, and lack of application thereof, makes it difficult to draw conclusions on which is the best-performing system for each data extraction target. With this living review we aim to review the literature continually.

show abstract

Section: Resultsmentioning

confidence: 99%

Data extraction methods for systematic review (semi)automation: A living systematic review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…We measured the correlation strength based on the standard guideline of the Pearson correlation coefficient in the biomedical domain [ 20 ]. We evaluated different similarity approaches, including string similarity (Jaccard), term frequency (count vectorizer), and pretrained word embedding using GloVe, Google word2vec, and fastText.…”

Section: Resultsmentioning

confidence: 99%

“…In a proceeding, the authors presented a process of a PICO corpus at the individual element level and sentence level [ 31 ]. A hybrid approach of combining machine learning and rule-based methods was proposed for the identification of PICO sentences and individual elements in successive order [ 20 ]. Another study on PICO sentence extraction was carried out with a supervised distance supervision approach that capitalizes on a small labeled dataset to mitigate noise in distantly derived annotations [ 32 ].…”

Section: Introductionmentioning

confidence: 99%

Clinical Context–Aware Biomedical Text Summarization Using Deep Neural Network: Model Development and Validation

Afzal¹,

Alam²,

Malik³

et al. 2020

J Med Internet Res

View full text Add to dashboard Cite

Background Automatic text summarization (ATS) enables users to retrieve meaningful evidence from big data of biomedical repositories to make complex clinical decisions. Deep neural and recurrent networks outperform traditional machine-learning techniques in areas of natural language processing and computer vision; however, they are yet to be explored in the ATS domain, particularly for medical text summarization. Objective Traditional approaches in ATS for biomedical text suffer from fundamental issues such as an inability to capture clinical context, quality of evidence, and purpose-driven selection of passages for the summary. We aimed to circumvent these limitations through achieving precise, succinct, and coherent information extraction from credible published biomedical resources, and to construct a simplified summary containing the most informative content that can offer a review particular to clinical needs. Methods In our proposed approach, we introduce a novel framework, termed Biomed-Summarizer, that provides quality-aware Patient/Problem, Intervention, Comparison, and Outcome (PICO)-based intelligent and context-enabled summarization of biomedical text. Biomed-Summarizer integrates the prognosis quality recognition model with a clinical context–aware model to locate text sequences in the body of a biomedical article for use in the final summary. First, we developed a deep neural network binary classifier for quality recognition to acquire scientifically sound studies and filter out others. Second, we developed a bidirectional long-short term memory recurrent neural network as a clinical context–aware classifier, which was trained on semantically enriched features generated using a word-embedding tokenizer for identification of meaningful sentences representing PICO text sequences. Third, we calculated the similarity between query and PICO text sequences using Jaccard similarity with semantic enrichments, where the semantic enrichments are obtained using medical ontologies. Last, we generated a representative summary from the high-scoring PICO sequences aggregated by study type, publication credibility, and freshness score. Results Evaluation of the prognosis quality recognition model using a large dataset of biomedical literature related to intracranial aneurysm showed an accuracy of 95.41% (2562/2686) in terms of recognizing quality articles. The clinical context–aware multiclass classifier outperformed the traditional machine-learning algorithms, including support vector machine, gradient boosted tree, linear regression, K-nearest neighbor, and naïve Bayes, by achieving 93% (16127/17341) accuracy for classifying five categories: aim, population, intervention, results, and outcome. The semantic similarity algorithm achieved a significant Pearson correlation coefficient of 0.61 (0-1 scale) on a well-known BIOSSES dataset (with 100 pair sentences) after semantic enrichment, representing an improvement of 8.9% over baseline Jaccard similarity. Finally, we found a highly positive correlation among the evaluations performed by three domain experts concerning different metrics, suggesting that the automated summarization is satisfactory. Conclusions By employing the proposed method Biomed-Summarizer, high accuracy in ATS was achieved, enabling seamless curation of research evidence from the biomedical literature to use for clinical decision-making.

show abstract

“…In our earlier research [ 27 ] we conceptualized the P element as a relationship between the semantic UMLS groups Disorders and Group [ 28 ] as shown in Fig. 3 .…”

Section: Methodsmentioning

confidence: 99%

Combination of conditional random field with a rule based method in the extraction of PICO elements

Chabou

Iglewski

2018

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

BackgroundExtracting primary care information in terms of Patient/Problem, Intervention, Comparison and Outcome, known as PICO elements, is difficult as the volume of medical information expands and the health semantics is complex to capture it from unstructured information. The combination of the machine learning methods (MLMs) with rule based methods (RBMs) could facilitate and improve the PICO extraction. This paper studies the PICO elements extraction methods. The goal is to combine the MLMs with the RBMs to extract PICO elements in medical papers to facilitate answering clinical questions formulated with the PICO framework.MethodsFirst, we analyze the aspects of the MLM model that influence the quality of the PICO elements extraction. Secondly, we combine the MLM approach with the RBMs in order to improve the PICO elements retrieval process. To conduct our experiments, we use a corpus of 1000 abstracts.ResultsWe obtain an F-score of 80% for P element, 64% for the I element and 92% for the O element. Given the nature of the used training corpus where P and I elements represent respectively only 6.5 and 5.8% of total sentences, the results are competitive with previously published ones.ConclusionsOur study of the PICO element extraction shows that the task is very challenging. The MLMs tend to have an acceptable precision rate but they have a low recall rate when the corpus is not representative. The RBMs backed up the MLMs to increase the recall rate and consequently the combination of the two methods gave better results.

show abstract

PICO extraction by combining the robustness of machine-learning methods with the rule-based methods

Cited by 7 publications

References 7 publications

Data extraction methods for systematic review (semi)automation: A living systematic review

Data extraction methods for systematic review (semi)automation: A living systematic review

Clinical Context–Aware Biomedical Text Summarization Using Deep Neural Network: Model Development and Validation

Combination of conditional random field with a rule based method in the extraction of PICO elements

Contact Info

Product

Resources

About