Hy-NLI: a Hybrid system for Natural Language Inference

Kalouli, Aikaterini-Lida; Crouch, Richard; Paiva, Valéria de

doi:10.18653/v1/2020.coling-main.459

Cited by 17 publications

(11 citation statements)

References 38 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Parikh et al, 2016) 71.1 45.2 51.4 ESIM (Chen et al, 2017) 66.1 42.1 53.8 BERT (Devlin et al, 2019) 82.7 22.8 44.7 BERT+ (Yanaka et al, 2019a) 76.0 70.3 71.6 NeuralLog (ours) 91.4 93.9 93.4 shows higher accuracy with a 4.92 percentage point increase. In addition, our system's accuracy has a 3.8 percentage point increase than another hybrid system, Hy-NLI (Kalouli et al, 2020). The good performance proves that our framework for joint logic and neural reasoning can achieve state-of-art performance on inference and outperforms existing systems.…”

Section: Experiments Setupmentioning

confidence: 74%

NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning

Chen¹,

Gao²,

Moss³

2021

Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics

View full text Add to dashboard Cite

Deep learning (DL) based language models achieve high performance on various benchmarks for Natural Language Inference (NLI). And at this time, symbolic approaches to NLI are receiving less attention. Both approaches (symbolic and DL) have their advantages and weaknesses. However, currently, no method combines them in a system to solve the task of NLI. To merge symbolic and deep learning methods, we propose an inference framework called NeuralLog, which utilizes both a monotonicity-based logical inference engine and a neural network language model for phrase alignment. Our framework models the NLI task as a classic search problem and uses the beam search algorithm to search for optimal inference paths. Experiments show that our joint logic and neural inference system improves accuracy on the NLI task and can achieve state-of-art accuracy on the SICK and MED datasets.

show abstract

Section: Experiments Setupmentioning

confidence: 74%

NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning

Chen¹,

Gao²,

Moss³

2021

Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics

View full text Add to dashboard Cite

show abstract

“…Neural Network with Logic Components for NLI: Recent works (Kalouli et al, 2020;Chen et al, 2021;Feng et al, 2020) have started to combine neural networks with logic-based components. The work most related to ours is Feng et al (2020), which adapts ESIM (Chen et al, 2017b) to predict relations between tokens in a premise and hypothesis, and composes them to predict final inferential labels.…”

Section: Related Workmentioning

confidence: 99%

Neuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference

Feng¹,

Yang²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce a neuro-symbolic natural logic framework based on reinforcement learning with introspective revision. The model samples and rewards specific reasoning paths through policy gradient, in which the introspective revision algorithm modifies intermediate symbolic reasoning steps to discover reward-earning operations as well as leverages external knowledge to alleviate spurious reasoning and training inefficiency. The framework is supported by properly designed local relation models to avoid input entangling, which helps ensure the interpretability of the proof paths. The proposed model has built-in interpretability and shows superior capability in monotonicity inference, systematic generalization, and interpretability, compared to previous models on the existing datasets.

show abstract

“…The gap between LMs and humans motivates us to explore more effective ways to make more effective reasoning in NLU. Maybe neural symbolic models are solutions to FOL reasoning (Kalouli et al, 2020).…”

Section: Overall Diagnosismentioning

confidence: 99%

Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI

Tian¹,

Li²,

Chen³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Recently, language models (LMs) have achieved significant performance on many NLU tasks, which has spurred widespread interest for their possible applications in the scientific and social area. However, LMs have faced much criticism of whether they are truly capable of reasoning in NLU. In this work, we propose a diagnostic method for first-order logic (FOL) reasoning with a new proposed benchmark, LogicNLI. LogicNLI is an NLI-style dataset that effectively disentangles the target FOL reasoning from commonsense inference and can be used to diagnose LMs from four perspectives: accuracy, robustness, generalization, and traceability. Experiments on BERT, RoBERTa, and XLNet, have uncovered the weaknesses of these LMs on FOL reasoning, which motivates future exploration to enhance the reasoning ability.

show abstract

Hy-NLI: a Hybrid system for Natural Language Inference

Cited by 17 publications

References 38 publications

NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning

NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning

Neuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference

Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI

Contact Info

Product

Resources

About