A survey of noise reduction methods for distant supervision

Abstract

We survey recent approaches to noise reduction in distant supervision learning for relation extraction. We group them according to the principles they are based on: at-least-one constraints, topic-based models, or pattern correlations. Besides describing them, we illustrate the fundamental differences and attempt to give an outlook to potentially fruitful further research. In addition, we identify related work in sentiment analysis which could profit from approaches to noise reduction.

A Machine Learning Filter for the Slot Filling Task

“…In 2014, the top performing system, submitted by Stanford, achieved a recall of 0.277, a precision of 0.546 and a F1 score of 0.368 [5,7]. These results are somewhat similar to those submitted the previous year by the best performing system, Relation Factory, which achieved a recall of 0.332, precision of 0.425 and F1 score of 0.373 [4,8].…”

“…For most systems, the slot filling task is done in two main steps: candidate generation and candidate validation [8]. The candidate generation stage can be further decomposed into the following steps: (i) query expansion, (ii) document retrieval and (iii) candidate matching.…”

“…The retained documents are annotated with a name entity recognizer that associates a named-entity tag (or the tag OTHER if the word is not a named entity) to each word. In the candidate matching stage, candidates, e.g., sentences containing a reference to the query entity or to some item of its expansion, along with a named-entity tag of a potential slot filler type for a given relation, are retrieved [8]. For example, the sentence Barack Obama was born in Honolulu, Hawaii in 1961. would be tagged by the named entity tagger as follows: "Barack/PERSON Obama/PERSON was/OTHER born/OTHER in/OTHER Honolulu/GPE-CITY, Hawaii/GPE-STATE in/OTHER 1961/DATE."…”

“…However, those relations are not necessarily easy to map to a knowledge base [11]. Therefore, modern systems (e.g., [7,8,12]) use different techniques such as pattern bootstrapping and distant supervision, which do not require labeled datasets but extract pre-defined relations.…”

Infusion of Labeled Data into Distant Supervision for Relation Extraction

“…One of most crucial problems in distant supervision is the inherent errors in the automatically generated training data (Roth et al, 2013). Table 1 illustrates this problem with a toy example.…”