Enhancing Unsupervised Pretraining with External Knowledge for Natural Language Inference

Yang, Xiaoyu; Zhu, Xiaodan; Zhao, Huasha; Zhang, Qiong; Feng, Yufei

doi:10.1007/978-3-030-18305-9_38

Cited by 20 publications

(20 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Neural models focusing solely on the textual information (Wang and Jiang 2016a;Yang et al 2019) explore the sentence representations of premise structure and max pooling layers. Match-LSTM (Wang and Jiang 2016a) and Decomposable Attention (Parikh et al 2016) learn crosssentence correlations using attention mechanisms, where the former uses a asymmetric network structure to learn premise-attended representation of the hypothesis, and the latter a symmetric attention, to decompose the problem into sub-problems.…”

Section: Related Workmentioning

confidence: 99%

Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks

Kapanipathi¹,

Thost²,

Patel³

et al. 2020

AAAI

View full text Add to dashboard Cite

Textual entailment is a fundamental task in natural language processing. Most approaches for solving this problem use only the textual content present in training data. A few approaches have shown that information from external knowledge sources like knowledge graphs (KGs) can add value, in addition to the textual content, by providing background knowledge that may be critical for a task. However, the proposed models do not fully exploit the information in the usually large and noisy KGs, and it is not clear how it can be effectively encoded to be useful for entailment. We present an approach that complements text-based entailment models with information from KGs by (1) using Personalized PageRank to generate contextual subgraphs with reduced noise and (2) encoding these subgraphs using graph convolutional networks to capture the structural and semantic information in KGs. We evaluate our approach on multiple textual entailment datasets and show that the use of external knowledge helps the model to be robust and improves prediction accuracy. This is particularly evident in the challenging BreakingNLI dataset, where we see an absolute improvement of 5-20% over multiple text-based entailment models.

show abstract

Section: Related Workmentioning

confidence: 99%

Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks

Kapanipathi¹,

Thost²,

Patel³

et al. 2020

AAAI

View full text Add to dashboard Cite

show abstract

“…In COLIEE 2021, we have two relationships that need to be verified: entailment and non-entailment. Yang et al [27] showed that human-created knowledge can further complement the use of pre-training models, to achieve better NLI prediction. Based on the results of Yang et al [27], we have exploited the external knowledge of the Kadokawa thesaurus [28], to tackle Tasks 4 and 5.…”

Section: Statute Law Textual Entailmentmentioning

confidence: 99%

Legal Information Retrieval and Entailment Based on BM25, Transformer and Semantic Thesaurus Methods

Kim

Rabelo

Okeke

et al. 2022

Rev Socionetwork Strat

View full text Add to dashboard Cite

We describe the techniques applied by the University of Alberta (UA) team in the most recent Competition on Legal Information Extraction and Entailment (COLIEE 2021). We participated in retrieval and entailment tasks for both case law and statute law; we applied a transformer-based approach for the case law entailment task, an information retrieval technique based on BM25 for legal information retrieval, and a natural language inference mechanism using semantic knowledge applied to statute law texts. This competition included 25 teams from 14 countries; our case law entailment approach was ranked no. 4 in Task 2, the BM25 technique for legal information retrieval was ranked no. 3 in Task 3, and the natural language inference technique incorporating semantic information was ranked no. 4 in Task 4. The combination of the latter two techniques on Task 5 was ranked no. 2. We also performed error analysis of our system in Task 4, which provides some insight into current state-of-the-art and research priorities for future directions.

show abstract

“…NLI (Dagan et al, 2005;Iftene and Balahur-Dobrescu, 2007;MacCartney and Manning, 2008;MacCartney and Manning, 2009;MacCartney, 2009;Angeli and Manning, 2014;Bowman et al, 2015), also known as recognizing textual entailment (RTE), aims to model the logical relationships between two sentences, e.g., as a binary (entailment vs. non-entailment) or three-way classification (entailment, contradiction, and neutral). Recently deep learning algorithms have been proposed (Bowman et al, 2015;Chen et al, 2017a;Chen et al, 2017b;Chen et al, 2017c;Chen et al, 2018;Peters et al, 2018;Yoon et al, 2018;Kiela et al, 2018;Talman et al, 2018;Yang et al, 2019;Devlin et al, 2019). In this paper we will describe and evaluate our neural natural logic models on NLI.…”

Section: Natural Language Inferencementioning

confidence: 99%

Exploring End-to-End Differentiable Natural Logic Modeling

Feng

Zheng

Liu

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

Self Cite

View full text Add to dashboard Cite

We explore end-to-end trained differentiable models that integrate natural logic with neural networks, aiming to keep the backbone of natural language reasoning based on the natural logic formalism while introducing subsymbolic vector representations and neural components. The proposed model adapts module networks to model natural logic operations, which is enhanced with a memory component to model contextual information. Experiments show that the proposed framework can effectively model monotonicity-based reasoning, compared to the baseline neural network models without built-in inductive bias for monotonicity-based reasoning. Our proposed model shows to be robust when transferred from upward to downward inference. We perform further analyses on the performance of the proposed model on aggregation, showing the effectiveness of the proposed subcomponents on helping achieve better intermediate aggregation performance.

show abstract

Enhancing Unsupervised Pretraining with External Knowledge for Natural Language Inference

Cited by 20 publications

References 23 publications

Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks

Infusing Knowledge into the Textual Entailment Task Using Graph Convolutional Networks

Legal Information Retrieval and Entailment Based on BM25, Transformer and Semantic Thesaurus Methods

Exploring End-to-End Differentiable Natural Logic Modeling

Contact Info

Product

Resources

About