Temporal Reasoning in Natural Language Inference

Vashishtha, Siddharth; Poliak, Adam; Lal, Yash Kumar; Durme, Benjamin Van; White, Aaron Steven

doi:10.18653/v1/2020.findings-emnlp.363

Cited by 21 publications

(22 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple systems have been proposed as part of research into temporal ordering (Do et al, 2012;Moens and Leeuwenberg, 2017;Leeuwenberg and Moens, 2018;Meng and Rumshisky, 2018;Ning et al, 2018c;Han et al, 2019), duration prediction (Vashishtha et al, 2019) and other tasks. Our decision to use a textual entailment style follows recent work on natural language inference (Williams et al, 2017;Nie et al, 2020;Bhagavatula et al, 2020), which tends to not focus on time (for recent work on temporal NLI, see Vashishtha et al (2020)). Many have used distant supervision for temporal reasoning (Gusev et al, 2011;Ning et al, 2018a;.…”

Section: Related Workmentioning

confidence: 99%

Temporal Reasoning on Implicit Events from Distant Supervision

Zhou¹,

Richardson²,

Ning³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

We propose TRACIE, a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit eventsevents that are not mentioned explicitly in natural language text but can be inferred from it. This introduces a new challenge in temporal reasoning research, where prior work has focused on explicitly mentioned events. Human readers can infer implicit events via commonsense reasoning, resulting in a more comprehensive understanding of the situation and, consequently, better reasoning about time. We find, however, that state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events. To address this, we propose a neuro-symbolic temporal reasoning model, SYMTIME, which exploits distant supervision signals from largescale text and uses temporal rules to combine start times and durations to infer end times. SYMTIME outperforms strong baseline systems on TRACIE by 5%, and by 11% in a zero prior knowledge training setting. Our approach also generalizes to other temporal reasoning tasks, as evidenced by a gain of 1%-9% on MATRES, an explicit event benchmark. * Most of the work was done when the third author was employed at the Allen Institute for AI and the first author was an intern there.Farrah was driving home from school. A person was riding a bicycle in front of her. Farrah looked away for a second. She didn't notice that he stopped. She tried to brake but it was too late. The person recovered soon. Context Story Latent Timeline ride stopped get hit injured A person drive hit try regret get home distracted explicit events implicit events not-inferrable distracted starts before try ✅ entailment distracted ends after try ❌ contradiction recovered Tracie Instance

show abstract

Section: Related Workmentioning

confidence: 99%

Temporal Reasoning on Implicit Events from Distant Supervision

Zhou¹,

Richardson²,

Ning³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…The dataset contains over 1.1K test instances. Each dialog contains 11.7 turns and 3 temporal expressions on average, presenting richer and more complex context compared to the recent single-sentence-based temporal question answering benchmarks (e.g., Zhou et al, 2019;Vashishtha et al, 2020). As above, each test instance contains two correct answers and two incorrect ones.…”

Section: Properties Of Timedialmentioning

confidence: 99%

“…Although previous works have studied temporal reasoning in natural language, they have either focused on specific time-related concepts in isolation, such as temporal ordering and relation extraction (Leeuwenberg and Moens, 2018;Ning et al, 2018a), and/or dealt with limited context, such as single-sentence-based question answering (Zhou et al, 2019) and natural language inference (Vashishtha et al, 2020;Mostafazadeh et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

TIMEDIAL: Temporal Commonsense Reasoning in Dialog

Qin¹,

Gupta²,

Upadhyay³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Everyday conversations require understanding everyday events, which in turn, requires understanding temporal commonsense concepts interwoven with those events. Despite recent progress with massive pre-trained language models (LMs) such as T5 and GPT-3, their capability of temporal reasoning in dialogs remains largely under-explored. In this paper, we present the first study to investigate pre-trained LMs for their temporal reasoning capabilities in dialogs by introducing a new task and a crowd-sourced English challenge set, TIMEDIAL. We formulate TIME-DIAL as a multiple choice cloze task with over 1.1K carefully curated dialogs. Empirical results demonstrate that even the best performing models struggle on this task compared to humans, with 23 absolute points of gap in accuracy. Furthermore, our analysis reveals that the models fail to reason about dialog context correctly; instead, they rely on shallow cues based on existing temporal patterns in context, motivating future research for modeling temporal concepts in text and robust contextual reasoning about them. The dataset is publicly available at: https://github.com/ google-research-datasets/timedial.

show abstract

“…We follow recent work that test for an expanded range of inference patterns in RTE systems (Bernardy and Chatzikyriakidis, 2019) by evaluating how well RTE models capture specific linguistic phenomena, such as pragmatic inferences (Jeretic et al, 2020), veridicality , and others (Pavlick and Callison-Burch, 2016;White et al, 2017;Dasgupta et al, 2018;Naik et al, 2018;Glockner et al, 2018;Kim et al, 2019;Kober et al, 2019;Richardson et al, 2020;Yanaka et al, 2020;Vashishtha et al, 2020;Poliak, 2020).…”

Section: Related Workmentioning

confidence: 99%

Figurative Language in Recognizing Textual Entailment

Chakrabarty¹,

Ghosh²,

Poliak³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

Self Cite

View full text Add to dashboard Cite

We introduce a collection of recognizing textual entailment (RTE) datasets focused on figurative language. We leverage five existing datasets annotated for a variety of figurative language -simile, metaphor, and irony -and frame them into over 12,500 RTE examples.We evaluate how well state-of-the-art models trained on popular RTE datasets capture different aspects of figurative language. Our results and analyses indicate that these models might not sufficiently capture figurative language, struggling to perform pragmatic inference and reasoning about world knowledge. Ultimately, our datasets provide a challenging testbed for evaluating RTE models.

show abstract

Temporal Reasoning in Natural Language Inference

Cited by 21 publications

References 50 publications

Temporal Reasoning on Implicit Events from Distant Supervision

Temporal Reasoning on Implicit Events from Distant Supervision

TIMEDIAL: Temporal Commonsense Reasoning in Dialog

Figurative Language in Recognizing Textual Entailment

Contact Info

Product

Resources

About