Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning

Xia, Jiangnan; Wu, Chen; Yan, Ming

doi:10.1145/3357384.3358165

Cited by 21 publications

(17 citation statements)

References 19 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…subgraphs from ConceptNet), and attending to it via attention mechanism when representing the inputs (Bauer et al, 2018;Paul and Frank, 2019;Lin et al, 2019). Alternative approaches include using the knowledge to score answer candidates and prune implausible ones (Lin et al, 2017;Tandon et al, 2018), and training in a multi-task setup via auxiliary tasks pertaining to knowledge (Xia et al, 2019).…”

Section: External Knowledge In Neural Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Commonsense Question Answering with Self-Talk

Shwartz¹,

West²,

Bras³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

128

160

View full text Add to dashboard Cite

Natural language understanding involves reading between the lines with implicit background knowledge. Current systems either rely on pretrained language models as the sole implicit source of world knowledge, or resort to external knowledge bases (KBs) to incorporate additional relevant knowledge. We propose an unsupervised framework based on self-talk as a novel alternative to multiple-choice commonsense tasks. Inspired by inquiry-based discovery learning (Bruner, 1961), our approach inquires language models with a number of information seeking questions such as "what is the definition of ..." to discover additional background knowledge. Empirical results demonstrate that the self-talk procedure substantially improves the performance of zeroshot language model baselines on four out of six commonsense benchmarks, and competes with models that obtain knowledge from external KBs. While our approach improves performance on several benchmarks, the selftalk induced knowledge even when leading to correct answers is not always seen as helpful by human judges, raising interesting questions about the inner-workings of pre-trained language models for commonsense reasoning.

show abstract

Section: External Knowledge In Neural Modelsmentioning

confidence: 99%

“…To increase the coverage of high-precision world knowledge and facilitate multi-hop reasoning by making intermediate reasoning steps explicit, prior work incorporated KBs (e.g. ConceptNet; Speer and Havasi, 2012) and knowledge-informed models into LM-based models (Xia et al, 2019;.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Commonsense Question Answering with Self-Talk

Shwartz¹,

West²,

Bras³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

128

160

View full text Add to dashboard Cite

show abstract

“…Rajani et al (2019) incorporates the generated explanations into the training of language models for enhancement. Xia et al (2019) leverages two auxiliary relation-aware tasks to better model the interactions between question and candidate answers. Chalier et al (2020) proposes a multi-faceted model of commonsense knowledge statements to capture more expressive meta-properties.…”

Section: Commonsense Reasoning Methodsmentioning

confidence: 99%

Beyond Language: Learning Commonsense from Images for Reasoning

Cui¹,

Lan

Pang

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

This paper proposes a novel approach to learn commonsense from images, instead of limited raw texts or costly constructed knowledge bases, for the commonsense reasoning problem in NLP. Our motivation comes from the fact that an image is worth a thousand words, where richer scene information could be leveraged to help distill the commonsense knowledge, which is often hidden in languages. Our approach, namely Loire, consists of two stages. In the first stage, a bi-modal sequence-tosequence approach is utilized to conduct the scene layout generation task, based on a text representation model ViBERT. In this way, the required visual scene knowledge, such as spatial relations, will be encoded in ViBERT by the supervised learning process with some bimodal data like COCO. Then ViBERT is concatenated with a pre-trained language model to perform the downstream commonsense reasoning tasks. Experimental results on two commonsense reasoning problems, i.e. commonsense question answering and pronoun resolution, demonstrate that Loire outperforms traditional language-based methods. We also give some case studies to show what knowledge is learned from images and explain how the generated scene layout helps the commonsense reasoning process.

show abstract

“…For applications without available structured knowledge bases, researchers have relied on commonsense aggregated from corpus statistics pulled from unstructured text (Tandon et al, 2018;Lin et al, 2017;Li et al, 2018;Banerjee et al, 2019). More recently, rather than providing relevant commonsense as an additional input to neural networks, researchers have looked into indirectly encoding commonsense knowledge into the parameters of neural networks through pretraining on commonsense knowledge bases (Zhong et al, 2018) or explanations (Rajani et al, 2019), or by using multi-task objectives with commonsense relation prediction (Xia et al, 2019).…”

Section: Descriptionmentioning

confidence: 99%