A new algorithmic approach for the extraction of temporal associations from clinical narratives with an application to medical product safety surveillance reports

Abstract

The sheer volume of textual information that needs to be reviewed and analyzed in many clinical settings requires the automated retrieval of key clinical and temporal information. The existing natural language processing systems are often challenged by the low quality of clinical texts and do not demonstrate the required performance. In this study, we focus on medical product safety report narratives and investigate the association of the clinical events with appropriate time information. We developed a novel algorithm for tagging and extracting temporal information from the narratives, and associating it with related events. The proposed algorithm minimizes the performance dependency on text quality by relying only on shallow syntactic information and primitive properties of the extracted event and time entities. We demonstrated the effectiveness of the proposed algorithm by evaluating its tagging and time assignment capabilities on 140 randomly selected reports from the US Vaccine Adverse Event Reporting System (VAERS) and the FDA (Food and Drug Administration) Adverse Event Reporting System (FAERS). We compared the performance of our tagger with the SUTime and HeidelTime taggers, and our algorithm's event-time associations with the Temporal Awareness and Reasoning Systems for Question Interpretation (TARSQI). We further evaluated the ability of our algorithm to correctly identify the time information for the events in the 2012 Informatics for Integrating Biology and the Bedside (i2b2) Challenge corpus. For the time tagging task, our algorithm performed better than the SUTime and the HeidelTime taggers (F-measure in VAERS and FAERS: Our algorithm: 0.86 and 0.88, SUTime: 0.77 and 0.74, and HeidelTime 0.75 and 0.42, respectively). In the event-time association task, our algorithm assigned an inappropriate timestamp for 25% of the events, while the TARSQI toolkit demonstrated a considerably lower performance, assigning inappropriate timestamps in 61.5% of the same events. Our algorithm also supported the correct calculation of 69% of the event relations to the section time in the i2b2 testing set.

Application of Natural Language Processing and Network Analysis Techniques to Post-market Reports for the Evaluation of Dose-related Anti-Thymocyte Globulin Safety Patterns

“…MedEx was previously found to perform well at identifying drug names and dose-related information, such as strength, route, and frequency [8,16,17]. ETHER's text mining capabilities have been extensively evaluated as well with a demonstrated efficiency in the extraction of drug names and temporal information [10,18]. The NLP output was subsequently analyzed in PANACEA, a NA tool that supports the analysis of complex relationships in large datasets over time.…”

“…For example, the reviewers concluded that no TSD should have been assigned by ETHER to many of the drug mentions. However, this should not characterize ETHER's ability to support this task as it is related to its particular configuration: if a drug is not directly linked to an absolute time statement in the text, ETHER will (and has been set to) assign an exposure date according to either the submission date or other time information in the narrative [10]. ETHER's configuration to avoid missing values can be easily changed to accommodate the requirements in other tasks.…”

“…ETHER is a rule-based NLP system that extracts clinical features and temporal associations from post-market safety surveillance reports as well as provides case summarization and information visualization [9]. ETHER's ability to extract time information and identify temporal relationships in FAERS narratives was previously demonstrated by Wang et al and is therefore an appropriate tool for generating missing TSD values [10].…”

Time Expressions in Mental Health Records for Symptom Onset Extraction

“…When applied to the TempEval-2 newspaper data, both systems achieved stateof-the-art performance (F1 scores of 92% and 86%, respectively, for time expression identification) (Verhagen et al, 2010). Moreover, they have previously been used for the automatic processing of clinical narratives (Jindal and Roth, 2013;Wang et al, 2016).…”

Advancing Public Health Using Regulatory Science to Enhance Development and Regulation of Medical Products: Food and Drug Administration Research at the Center for Biologics Evaluation and Research

“…The current natural language processing systems are often hampered by low quality text (e.g., poor grammar, many abbreviations). CBER developed an algorithm for analyzing low quality texts that is minimally dependent on grammatical and syntactic information (30). This will speed reviewer extraction and analysis of important information, and accelerate recommendations about post-marketing adverse events related to medical product use.…”