Commonsense mining as knowledge base completion? A study on the impact of novelty

Jastrzębski, Stanisław; Bahdanau, Dzmitry; Hosseini, S. Jafar; Noukhovitch, Michael; Bengio, Yoshua; Cheung, Jackie Kit

doi:10.48550/arxiv.1804.09259

Cited by 1 publication

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Methodsmentioning

confidence: 99%

“…We compare our results to the supervised models from the work of Jastrzębski et al (2018) and the best performing model from Li et al (2016). Jastrzębski et al (2018) introduce FAC-TORIZED and PROTOTYPICAL models. The Factorized model embeds the head, relation, and tail in a vector space and then produces a score by taking a linear combination of the inner products between each pair of embeddings.…”

Section: Methodsmentioning

confidence: 99%

“…Several approaches have been proposed for training models for commonsense knowledge base completion (Li et al, 2016;Jastrzębski et al, 2018). Each of these approaches uses some sort of supervised training on a particular knowledge base, evaluating the model's performance on a held-out test set from the same database.…”

Section: Introductionmentioning

confidence: 99%

“…These works use relations from ConceptNet, a crowd-sourced database of structured commonsense knowledge, to train and validate their models (Liu and Singh, 2004). However, it has been shown that these methods generalize poorly to novel data (Li et al, 2016;Jastrzębski et al, 2018). Jastrzębski et al (2018) demonstrated that much of the data in the ConceptNet test set were simply rephrased relations from the training set, and that this train-test set leakage led to artificially inflated test performance metrics.…”

Section: Introductionmentioning

confidence: 99%

“…However, it has been shown that these methods generalize poorly to novel data (Li et al, 2016;Jastrzębski et al, 2018). Jastrzębski et al (2018) demonstrated that much of the data in the ConceptNet test set were simply rephrased relations from the training set, and that this train-test set leakage led to artificially inflated test performance metrics. This problem of traintest leakage is typical in knowledge base completion tasks (Toutanova et al, 2015;Dettmers et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Commonsense Knowledge Mining from Pretrained Models

Feldman¹,

Js²,

Rush³

2019

Preprint

View full text Add to dashboard Cite

Inferring commonsense knowledge is a key challenge in natural language processing, but due to the sparsity of training data, previous work has shown that supervised methods for commonsense knowledge mining underperform when evaluated on novel data. In this work, we develop a method for generating commonsense knowledge using a large, pre-trained bidirectional language model. By transforming relational triples into masked sentences, we can use this model to rank a triple's validity by the estimated pointwise mutual information between the two entities. Since we do not update the weights of the bidirectional model, our approach is not biased by the coverage of any one commonsense knowledge base. Though this method performs worse on a test set than models explicitly trained on a corresponding training set, it outperforms these methods when mining commonsense knowledge from new sources, suggesting that unsupervised techniques may generalize better than current supervised approaches.

show abstract

Section: Methodsmentioning

confidence: 99%