CoType

Abstract

Extracting entities and relations for types of interest from text is important for understanding massive text corpora. Traditionally, systems of entity relation extraction have relied on human-annotated corpora for training and adopted an incremental pipeline. Such systems require additional human expertise to be ported to a new domain, and are vulnerable to errors cascading down the pipeline. In this paper, we investigate joint extraction of typed entities and relations with labeled data heuristically obtained from knowledge bases (i.e., distant supervision). As our algorithm for type labeling via distant supervision is context-agnostic, noisy training data poses unique challenges for the task. We propose a novel domainindependent framework, called COTYPE, that runs a data-driven text segmentation algorithm to extract entity mentions, and jointly embeds entity mentions, relation mentions, text features and type labels into two low-dimensional spaces (for entity and relation mentions respectively), where, in each space, objects whose types are close will also have similar representations. COTYPE, then using these learned embeddings, estimates the types of test (unlinkable) mentions. We formulate a joint optimization problem to learn embeddings from text corpora and knowledge bases, adopting a novel partial-label loss function for noisy labeled data and introducing an object "translation" function to capture the cross-constraints of entities and relations on each other. Experiments on three public datasets demonstrate the effectiveness of COTYPE across different domains (e.g., news, biomedical), with an average of 25% improvement in F1 score compared to the next best method.

Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach

“…CoType-RM (Ren et al, 2016) adopts partial-label loss to handle label noise and train the extractor. Moreover, two different strategies are adopted to feed heterogeneous supervision to these methods.…”

“…As shown in Table 2, we extract abundant lexical features (Ren et al, 2016;Mintz et al, 2009) to characterize relation mentions. However, this abundance also results in the gigantic dimension of original text features (∼ 10 7 in our case).…”

“…Additionally, not like other relation types in R, None does not have a coherent semantic meaning. Similar to (Ren et al, 2016), we introduce a heuristic rule: identifying a relation mention as None when (1) our relation extractor predict it as None, or (2) the entropy of p(.|zc) over R exceeds a pre-defined threshold η. The entropy is calculated as…”

“…One is knowledge base, the other is handcrafted domain-specific patterns. For domain-specific patterns, we manually design a number of labeling functions 3 ; for knowledge base, annotations are generated following the procedure in (Ren et al, 2016;Riedel et al, 2010).…”

“…To alleviate the dependency of annotations given by human experts, weak supervision (Bunescu and Mooney, 2007;Etzioni et al, 2004) and distant supervision (Ren et al, 2016) have been employed to automatically generate annotations based on knowledge base (or seed patterns/instances). Universal Schemas (Riedel et al, 2013;Verga et al, 2015;Toutanova et al, 2015) has been proposed to unify patterns and knowledge base, but it's designed for document-level relation extraction, i.e., not to categorize relation types based on a specific context, but based on the whole corpus.…”