Discrete-State Variational Autoencoders for Joint Discovery and Factorization of Relations

Abstract

We present a method for unsupervised open-domain relation discovery. In contrast to previous (mostly generative and agglomerative clustering) approaches, our model relies on rich contextual features and makes minimal independence assumptions. The model is composed of two parts: a feature-rich relation extractor, which predicts a semantic relation between two entities, and a factorization model, which reconstructs arguments (i.e., the entities) relying on the predicted relation. The two components are estimated jointly so as to minimize errors in recovering arguments. We study factorization models inspired by previous work in relation factorization and selectional preference modeling. Our models substantially outperform the generative and agglomerative-clustering counterparts and achieve state-of-the-art performance.

Quantifying Similarity between Relations with Fact Distribution

“…11The detailed instruction is attached in the Appendix F. Recently, some efforts are devoted to Open Relation Extraction (Open RE) (Lin and Pantel, 2001;Yao et al, 2011;Marcheggiani and Titov, 2016;ElSahar et al, 2017), aiming to cluster relation patterns into several relation types instead of redundant relation patterns. Whenas, these Open RE methods adopt distantly supervised labels as golden relation types, suffering from both false positive and false negative problems on the one hand.…”

Relation Discovery with Out-of-Relation Knowledge Base as Supervision

“…We evaluate our model in the context of unsupervised relation discovery and compare to the baseline model, DVAE (Marcheggiani and Titov, 2016) which is the current state-of-the-art of relation discovery. Distant supervision assumes that the relations should be aligned between the KB and the training text corpus, which is not available in our setting.…”

“…We tested our model on three different subsets of New York Times corpus (NYT) (Sandhaus and Evan, 2008). • The first one is widely used in unsupervised settings, which was developed by Yao et al (2011) and has also been used by Marcheggiani and Titov (2016). This dataset contains articles 2000 to 2007, with named entities annotated and features processed (POS tagging, NER, and syntactic parsing).…”

“…The statistics of the three datasets are shown in Table 2. For NYT71 and NYT27, we perform the same feature extraction as NYT122 shown in (Marcheggiani and Titov, 2016).…”

“…If the KB tells us these two pairs of tuples are close to each other, then we put a constraint to force our relation clustering algorithm to group them together. We use the discrete-state variational autoencoder (DVAE) framework (Marcheggiani and Titov, 2016) as our base relation discovery model since this framework is flexible to incorporate different features and currently the state-of-the-art. We use KB embedding (Bordes et al, 2013) to obtain entity embeddings in the KB and use entity embeddings to evaluate the similarity between a pair of tuples.…”