End-to-End Fine-Grained Neural Entity Recognition of Patients, Interventions, Outcomes

Abstract

PICO recognition is an information extraction task for detecting parts of text describing Participant (P), Intervention (I), Comparator (C), and Outcome (O) (PICO elements) in clinical trial literature. Each PICO description is further decomposed into finer semantic units. For example, in the sentence 'The study involved 242 adult men with back pain.', the phrase '242 adult men with back pain' describes the participant, but this coarse-grained description is further divided into finer semantic units. The term '242' shows "sample size" of the participants, 'adult' shows "age", 'men' shows "sex", and 'back pain' show the participant "condition". Recognizing these fine-grained PICO entities in health literature is a challenging named-entity recognition (NER) task but it can help to fully automate systematic reviews (SR). Previous approaches concentrated on coarse-grained PICO recognition but focus on the fine-grained recognition still lacks. We revisit the previously unfruitful neural approaches to improve recognition performance for the fine-grained entities. In this paper, we test the feasibility and quality of multitask learning (MTL) to improve fine-grained PICO recognition using a related auxiliary task and compare it with single-task learning (STL). As a consequence, our end-to-end neural approach improves the state-of-the-art (SOTA) F1 score from 0.45 to 0.54 for the "participant" entity and from 0.48 to 0.57 for the "outcome" entity without any handcrafted features. We inspect the models to identify where they fail and how some of these failures are linked to the current benchmark data.

Towards precise PICO extraction from abstracts of randomized controlled trials using a section-specific learning approach

“…One such issue was concerned with the “Intervention” element. According to Dhrangadhariya et al (2021) , even the human annotators find it difficult to classify certain interventions as education or psychological. Their experiments showed the least performance of 0.31 F1-score on the “Intervention” class.…”

“…NER is used to identify the PICO elements ( Nguyen et al 2017 , Nye et al 2018 , Kang et al 2019 , 2021 , Zhang et al 2020 , Dhrangadhariya et al 2021 , Liu et al 2021b ). Nye et al (2018) presented two baseline models, the linear CRF model and the LSTM-CRF model for identifying PICO elements in the EBM-NLP corpus.…”

“…A multi-task learning approach that learns and recognizes both coarse-grained descriptions (e.g. 40 children aged 7–11 with autism spectrum disorder) and constituent finer semantic units (e.g., “40” shows “sample size”, “7–11” shows “age” and “autism spectrum disorder” shows “condition”) was explored by Dhrangadhariya et al (2021) . In that study, the EBM-NLP corpus was utilized as it provided a multi-level annotation: the span-level (level 1) annotation corresponds to the coarse-grained descriptions and other levels of annotation focus on specific semantic units.…”

DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resource Entity Extraction Using Clinical Trials Literature

“…We evaluate our weakly annotated dataset and the NER model on the following PICO benchmarks. ) was used as an additional benchmark to evaluate the generalization power of our approach for this subdomain (Dhrangadhariya et al, 2021).…”

“…', it will remain unmapped to it because the term does not map to the target text using our alignment heuristic. The problem lies in the lack of naming conventions for non-pharma treatment mentions that are neither clearly identified nor standardized as semantic units (Dhrangadhariya et al, 2021). There are two possible programmatic solutions to this.…”