TDDiscourse: A Dataset for Discourse-Level Temporal Ordering of Events

Abstract

Prior work on temporal relation classification has focused extensively on event pairs in the same or adjacent sentences (local), paying scant attention to discourse-level (global) pairs. This restricts the ability of systems to learn temporal links between global pairs, since reliance on local syntactic features suffices to achieve reasonable performance on existing datasets. However, systems should be capable of incorporating cues from documentlevel structure to assign temporal relations. In this work, we take a first step towards discourse-level temporal ordering by creating TDDiscourse, the first dataset focusing specifically on temporal links between event pairs which are more than one sentence apart. We create TDDiscourse by augmenting TimeBank-Dense, a corpus of English news articles, manually annotating global pairs that cannot be inferred automatically from existing annotations. Our annotations double the number of temporal links in TimeBank-Dense, while possessing several desirable properties such as focusing on long-distance pairs and not being automatically inferable. We adapt and benchmark the performance of three stateof-the-art models on TDDiscourse and observe that existing systems indeed find discourselevel temporal ordering harder.

Distinguishing Between Foreground and Background Events in News

“…We extracted and resolved the arguments of events using AllenNLP's semantic role labeling and coreference resolution . We train a pairwise logistic regression classifier from scikit-learn over the features using parameters shown in (Naik et al, 2019). The last two rows show our model without and with Foreground/Background fine-grained features, respectively.…”

“…In this experiment, we target the extraction of temporal relation between events which is one of the fundamental tasks in temporal processing as identified in the series TempEval (TE) workshops (Verhagen et al, 2007;Verhagen et al, 2010;UzZaman et al, 2013). We used the recently published dataset TDDiscourse (Naik et al, 2019), an augmented dataset of TimeBank-Dense (Cassidy et al, 2014) focused on discourse-level temporal ordering and used the same set of temporal relations as TimeBank-Dense (i.e., after, before, simultaneous , includes and is-included). The annotation of the corpus consists of two sets: Manual annotation (TDD-Man) and Automatic inference (TDD-Auto), we experiment on both.…”

“…As shown in Table 7, our approach in general outperforms all models on both TDD-auto and TDD-man by 9% and 5%, respectively. The reason behind the low performance of the other models has been addressed in Naik et al (2019), and is out of scope for this paper. With regard to our model, as shown in the table, adding Foreground/Background fine-grained features did not help much in improving the model performance.…”

“…1 Introduction Grimes et al (1975) defined foreground events as the events that form the skeleton of a story whereas background events add supporting information. Ability to automatically extract such a distinction could guide document understanding and potentially be helpful in many natural language processing tasks such as temporal relation extraction (Naik et al, 2019) , summarization (Zhang et al, 2018), and storyline generation . We introduce the task of distinguishing between foreground and background events, as well as identifying the general temporal position of backgrounds events relative to the foreground period (past, present, future, and their combinations).…”

Temporal Reasoning in Natural Language Inference

“…Natural language supports various forms of temporal reasoning, including reasoning about the chronology and duration of events, and many Natural Language Understanding (NLU) tasks and models have been employed for understanding and capturing different aspects of temporal reasoning (UzZaman et al, 2013;Llorens et al, 2015;Mostafazadeh et al, 2016;Reimers et al, 2016;Tourille et al, 2017;Ning et al, 2017Ning et al, , 2018aMeng and Rumshisky, 2018;Ning et al, 2018b;Han et al, 2019;Naik et al, 2019;Vashishtha et al, 2019;Zhou et al, 2019Zhou et al, , 2020. More broadly, the ability to perform temporal reasoning is important for understanding narratives (Nakhimovsky, 1987;Jung et al, 2011;Cheng et al, 2013), answering questions (Bruce, 1972;Khashabi, 2019;, and summarizing events (Jung et al, 2011;Wang et al, 2018).…”