ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning

Han, Rujun; Hsu, I-Hung; Sun, Jiao; Baylon, Julia; Ning, Qiang; Roth, Dan; Peng, Nanyun

doi:10.48550/arxiv.2104.08350

Cited by 4 publications

(4 citation statements)

References 27 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most recently, ESTER was introduced [6], which is an MRC dataset for Event Semantic Relation Reasoning. The dataset contains natural language queries to reason about the five most common event semantic relations.…”

Section: Related Workmentioning

confidence: 99%

“…As HuRC was created mainly automatically, the chance of erroneous labels or masking is certainly high. We aimed to provide a test set as clean and accurate as possible, therefore the 8 000 instances of the test set were manually validated again against the following criteria: i) whether the named entity recognition and masking was correct, 6 ii) whether each and every named entity in the passage is listed in the list of named entities found by the NER model. This manual validation required >100 work hours of an annotator.…”

Section: The Test Setmentioning

confidence: 99%

See 1 more Smart Citation

Building machine reading comprehension model from scratch

Yang¹,

Ligeti-Nagy²

2023

AMI

View full text Add to dashboard Cite

In this paper, we introduce a machine reading comprehension model and how we built this model from scratch. Reading comprehension is a crucial requisite for artificial intelligence applications, such as Question-Answering systems, chatbots, virtual assistants etc. Reading comprehension task requires the highest complexity of natural language processing methods. In recent years, the transformer neural architecture could achieve the ability to solve high complexity tasks. To make these applications available in Hungarian as well it is inevitable to design a Hungarian corpus of reading comprehension so that the pretrained models can be fine-tuned on this dataset.In our research, we have created the HuRC (Hungarian Reading Comprehension) corpus, which is the first dataset in Hungarian aiming to train, test and evaluate language models on a reading comprehension task. We built such a dataset based on the English ReCoRD corpus. This is a dataset of 120,000 examples consisting of news articles containing a passage and a close-style query, where a named entity is masked and the reference answer has to be found in a list.Using the evaluated dataset and transformers' question-answering library, we have built the first neural machine reading comprehension models in commonsense reasoning task for Hungarian.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: The Test Setmentioning

confidence: 99%

Building machine reading comprehension model from scratch

Yang¹,

Ligeti-Nagy²

2023

AMI

View full text Add to dashboard Cite

show abstract

“…As a significant step towards inducing event complexes (graphs that recognize the relationship of multi-granular events) in documents, subevent detection has started to receive attention recently Han et al, 2021). It is natu-ral to perceive that in documents, there might be several different event complexes and they often span in different descriptive contexts that form relatively independent text segments.…”

Section: Introductionmentioning

confidence: 99%

Learning Constraints and Descriptive Segmentation for Subevent Detection

Wang

Zhang

Chen

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Event mentions in text correspond to realworld events of varying degrees of granularity. The task of subevent detection aims to resolve this granularity issue, recognizing the membership of multi-granular events in event complexes. Since knowing the span of descriptive contexts of event complexes helps infer the membership of events, we propose the task of event-based text segmentation (EVENTSEG) as an auxiliary task to improve the learning for subevent detection. To bridge the two tasks together, we propose an approach to learning and enforcing constraints that capture dependencies between subevent detection and EVENTSEG prediction, as well as guiding the model to make globally consistent inference. Specifically, we adopt Rectifier Networks for constraint learning and then convert the learned constraints to a regularization term in the loss function of the neural model. Experimental results show that the proposed method outperforms baseline methods by 2.3% and 2.5% on benchmark datasets for subevent detection, HiEve and IC, respectively, while achieving a decent performance on EVENTSEG prediction 1 .

show abstract

“…In contrast, plot comprehension is a popular NLP topic, especially on event structures(Finlayson, 2012;Elsner, 2012;Sims et al, 2019;Lal et al, 2021;Han et al, 2021).…”

mentioning

confidence: 99%

TVShowGuess: Character Comprehension in Stories as Speaker Guessing

Sang¹,

Mou²,

Yu³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

We propose a new task for assessing machines' ability to understand fictional characters in narrative stories. The task, TVSHOWGUESS, builds on the scripts of TV series and takes the form of guessing the anonymous main characters based on the backgrounds of the scenes and dialogues. Our human study supports that this form of task covers comprehension of multiple types of character persona, including understanding characters' personalities, facts and memories of personal experience, which are well aligned with the psychological and literary theories about the theory of mind (ToM) of human beings on understanding fictional characters during reading. We further propose new model architectures to support the contextualized encoding of long scene texts. Experiments show that our proposed approaches significantly outperform baselines, yet still largely lag behind the (nearly perfect) human performance. Our work serves as a first step toward the goal of narrative character comprehension. 1

show abstract

ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning

Cited by 4 publications

References 27 publications

Building machine reading comprehension model from scratch

Building machine reading comprehension model from scratch

Learning Constraints and Descriptive Segmentation for Subevent Detection

TVShowGuess: Character Comprehension in Stories as Speaker Guessing

Contact Info

Product

Resources

About