Zero-shot Event Extraction via Transfer Learning: Challenges and Insights

Abstract

Event extraction has long been a challenging task, addressed mostly with supervised methods that require expensive annotation and are not extensible to new event ontologies. In this work, we explore the possibility of zeroshot event extraction by formulating it as a set of Textual Entailment (TE) and/or Question Answering (QA) queries (e.g. "A city was attacked" entails "There is an attack"), exploiting pretrained TE/QA models for direct transfer. On ACE-2005 and ERE, our system achieves acceptable results, yet there is still a large gap from supervised approaches, showing that current QA and TE technologies fail in transferring to a different domain. To investigate the reasons behind the gap, we analyze the remaining key challenges, their respective impact, and possible improvement directions 1 .

ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizations

“…For RE, Wang et al (2021a) propose a text-to-triple translation method that given a text and a set of entities returns the existing relations. For EE, Lyu et al (2021) to us, the use of an entailment model, but in their case the input sentence is split in clauses according to the output of a Semantic Role Labelling system. In order to compare their results with ours, we only use the event types, not the trigger information 3 .…”

“…Similar to TE, (Clark et al, 2019) performs yes/no Question Answering, in which a model is asked about the veracity of some fact given a passage. Lyu et al (2021) recast the zero-shot event extraction as a TE task, using TE model to check whether a piece of text is about a type of event.…”

Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey

“…also experimented in a zero-shot setting where no task-specific data is used for training, only using prompts for probing. The zeroshot setting for the full event extraction pipeline has been explored in Lyu et al (2021) where QAbased prompts are used for argument extraction and prompts based on Textual Entailment (Dagan et al, 2013) are used for trigger classification (see Section 3.3.1 below). Several ablation experiments analyzed the different components of the system such as the choice of PLM, the choice of QA dataset and the way to generate the questions (fixed vs. contextualized).…”

“…Some of the works introduce contextualization, integrating relevant words from the text into the question. For example, in argument extraction, the question can include the trigger extracted from the text (e.g Lyu et al, 2021) or another argument that was previously identified (see the Event Extraction row in Table 4). Neural based question generation models can also improve the quality of the question, as in , where monolingual unsupervised machine translation is used to generate the part of the question that does not depend on the template, translating a descriptive statement into a question-style expression.…”

The Art of Prompting: Event Detection based on Type Specific Prompts

“…Event detection (Grishman, 1997;Chinchor and Marsh, 1998;Ahn, 2006) is the task of identifying and typing event mentions from natural language text. Supervised approaches, especially deep neural networks Du and Cardie, 2020;Lin et al, 2020;Lyu et al, 2021), have shown remarkable performance under a critical prerequisite of a large amount of manual annotations. However, they cannot be effectively generalized to new languages, domains or types, especially when the annotations are not enough Lai et al, 2020b;Shen et al, 2021) or there is no annotations available (Lyu et al, 2021;Zhang et al, 2021b;Pasupat and Liang, 2014).…”