MZET: Memory Augmented Zero-Shot Fine-grained Named Entity Typing

Abstract

Named entity typing (NET) is a classification task of assigning an entity mention in the context with given semantic types. However, with the growing size and granularity of the entity types, few previous researches concern with newly emerged entity types. In this paper, we propose MZET, a novel memory augmented FNET (Fine-grained NET) model, to tackle the unseen types in a zero-shot manner. MZET incorporates character-level, word-level, and contextural-level information to learn the entity mention representation. Besides, MZET considers the semantic meaning and the hierarchical structure into the entity type representation. Finally, through the memory component which models the relationship between the entity mention and the entity type, MZET transfers the knowledge from seen entity types to the zero-shot ones. Extensive experiments on three public datasets show the superior performance obtained by MZET, which surpasses the state-of-the-art FNET neural network models with up to 8% gain in Micro-F1 and Macro-F1 score.

An Empirical Study on Multiple Information Sources for Zero-Shot Fine-Grained Entity Typing

“…(1) Proto-HLE (Ma et al, 2016) which introduces prototype-driven hierarchical label embedding for ZFET; (2) ZOE (Zhou et al, 2018) which infers the types of a given mention according to its typecompatible Wikipedia entries; (3) DZET (Obeidat et al, 2019) which derives type representations from Wikipedia pages and leverages a contextdescription matching approach for type inference; (4) NZFET * (Ren et al, 2020) which employs entity type attention to make the model focus on information relevant to the entity type; (5) MZET * (Zhang et al, 2020b) which adopts a memory network to connect the seen and unseen types.…”

“…In many scenarios, the type hierarchy is continuously evolving, which requires newly emerged types to be accounted into FET systems. As a result, zero-shot FET (ZFET) is welcomed to handle the new types which are unseen during training stage (Ma et al, 2016;Ren et al, 2020;Zhang et al, 2020b).…”

“…The major challenge of ZFET is to build the semantic connections between the seen types (during training) and the unseen ones (during inference). * Corresponding Author Auxiliary information has been proved to be essential in this regard (Xian et al, 2019), with a variety of approaches focused on scattered information (Ma et al, 2016;Zhou et al, 2018;Obeidat et al, 2019;Ren et al, 2020;Zhang et al, 2020b). However, the power of auxiliary information has not been sufficiently exploited in existing solutions.…”

“…(ii) In the HA (Type-Hierarchy Aware) module, we use Transformer encoder (Vaswani et al, 2017) to model the hierarchical dependency among types. There have been substantial works exploring type hierarchy in the supervised typing task (Shimaoka et al, 2017;Xu and Barbosa, 2018;Xiong et al, 2019), but only some preliminary research in ZFET (Ma et al, 2016;Zhang et al, 2020b). (iii) In the KA (Background-Knowledge Aware) module, we introduce prototypes (Ma et al, 2016) and WordNet descriptions (Miller, 1995) as background knowledge of types.…”

Fine-Grained Entity Typing via Label Noise Reduction and Data Augmentation

“…The ultra fine-grained label set also leads to data bottleneck and the long tail problem. In recent years, some previous approaches try to tackle this problem by introducing zero/few-shot learning methods (Ma et al, 2016;Zhou et al, 2018;Yuan and Downey, 2018;Obeidat et al, 2019;Zhang et al, 2020b;, or using data augmentation with denosing strategies (Ren et al, 2016b;Onoe and Durrett, 2019;Zhang et al, 2020a;Ali et al, 2020) or utilizing external knowledge (Corro et al, 2015;Dai et al, 2019) to introduce more external knowledge.…”