Empirical Exploration of the Challenges in Temporal Relation Extraction from Clinical Text

Abstract

Time is an important concept in human-cognition, fundamental to a wide range of reasoning tasks in the clinical domain. Results of the Clinical TempEval 2016 challenge, a set of shared tasks that evaluate temporal information extraction systems in the clinical domain, indicate that current state-of-the-art systems do well in solving event and time expression identification but perform poorly in temporal relation extraction. This study aims to identify and analyze the reason(s) for this uneven performance. It adapts a general domain tree-based bidirectional long short-term memory recurrent neural network model for semantic relation extraction to the task of temporal relation extraction in the clinical domain, and tests the system in a binary and multi-class classification setting by experimenting with general and in-domain word embeddings. Its results outperform the best Clinical TempEval 2016 system and the current stateof-the-art model. However, there is still a significant gap between the system and human performance. Consequently, this study delivers a deep analysis of the results, identifying a high incidence of nouns as events and class overlapping as posing major challenges in this task.

Temporal Relation Extraction in Clinical Texts

“…Based on these results, we highlight [89] and [99]. The authors of [89] adapted the tree-based Bi-LSTM-RNN model in [133], making new sentence-level annotations to adapt the input, relying on the dependency structure between the pair and the output. The authors of [99] combined several factors that were successful in the previous approaches.…”

“…Among publications that addressed only within-sentence relations, the authors of [89] and [90] used tree-based Bi-LSTM-RNNs, the authors of [99] used BI-LSTM with self-training, the authors of [87] used CNNs, the authors of [101] used a hybrid approach based on a CNN and an SVM model, the authors of [104] used GRUs and attention, and the authors of [77] used RNNs, attention, and piecewise representation. Based on these results, we highlight [89] and [99]. The authors of [89] adapted the tree-based Bi-LSTM-RNN model in [133], making new sentence-level annotations to adapt the input, relying on the dependency structure between the pair and the output.…”

Clinical Temporal Relation Extraction with Probabilistic Soft Logic Regularization and Global Inference

“…Models. Some early efforts to solve the clinical relation extraction problem leverage conventional machine learning methods (Llorens, Saquete, and Navarro 2010;Sun, Rumshisky, and Uzuner 2013;Xu et al 2013;Tang et al 2013;Lee et al 2016;Chikka 2016) such as SVMs, Max-Ent and CRFs, and neural network based methods (Lin et al 2017(Lin et al , 2018Dligach et al 2017;Tourille et al 2017;Lin et al 2019;Guan et al 2020;Lin et al 2020;Galvan-Sosa et al 2020). They either require expensive feature engineering or fail to consider the dependencies among temporal relations within a document.…”