MAVEN: A Massive General Domain Event Detection Dataset

Abstract

Event detection (ED), which means identifying event trigger words and classifying event types, is the first and most fundamental step for extracting event knowledge from plain text. Most existing datasets exhibit the following issues that limit further development of ED: (1) Data scarcity. Existing smallscale datasets are not sufficient for training and stably benchmarking increasingly sophisticated modern neural methods. (2) Low coverage. Limited event types of existing datasets cannot well cover general-domain events, which restricts the applications of ED models. To alleviate these problems, we present a MAssive eVENt detection dataset (MAVEN), which contains 4, 480 Wikipedia documents, 118, 732 event mention instances, and 168 event types. MAVEN alleviates the data scarcity problem and covers much more general event types. We reproduce the recent state-of-the-art ED models and conduct a thorough evaluation on MAVEN. The experimental results show that existing ED methods cannot achieve promising results on MAVEN as on the small datasets, which suggests that ED in the real world remains a challenging task and requires further research efforts. We also discuss further directions for general domain ED with empirical analyses. The source code and dataset can be obtained from https:// github.com/THU-KEG/MAVEN-dataset.

Event Extraction from Historical Texts: A New Dataset for Black Rebellions

“…zero-shot learning (Huang et al, 2018), fewshot learning (Lai et al, 2020a,b), or new domains (Naik and Rosé, 2020). The closet works to ours involve recent efforts to create new datasets for EE (Satyapanich et al, 2020;Ebner et al, 2020;Wang et al, 2020;Trong et al, 2020;Le and Nguyen, 2021). However, these works do not consider historical texts as we do.…”

“…OneIE (Lin et al, 2020): This model first identifies spans of entity mentions and event triggers. The detected spans are then paired to jointly predict entity types, event types, relations, and argument roles for IE.…”

“…This working methodology requires an emphasis on the quality of the data over the quantity of the data. Recent advances of natural language processing (NLP) aim to bridge the gap between qualitative and quantitative analyses by identifying, extracting, and counting contextual data (Won et al, 2018;Wadden et al, 2019;Lin et al, 2020). This new approach provides contextual information about real-life entities (e.g., individuals, locations, times, documents) which can be later integrated into knowledge bases (Won et al, 2018) to aid historical research and discourse analysis.…”

OntoED: Low-resource Event Detection with Ontology Embedding

“…(1) Overall Evaluation, (2) Few-shot Evaluation, and (3) Zero-shot Evaluation. OntoEvent is established based on two newly proposed datasets for ED: MAVEN (Wang et al, 2020b) and FewEvent (Deng et al, 2020). They are constructed from Wikipedia documents or based on existing event datasets, such as ACE-2005 5 and TAC-KBP-2017 6 .…”

“…As a non-trivial task, ED suffers from the lowresource issues. On the one hand, the maldistribu- tion of samples is quite serious in ED benchmark datasets, e.g., FewEvent (Deng et al, 2020) and MAVEN (Wang et al, 2020b), where a large portion of event types contain relatively few training instances. As shown in Figure 1, the sample size of two event types Attack and Riot differs greatly (4816 & 30).…”