Exploring End-to-End Differentiable Natural Logic Modeling

Abstract

We explore end-to-end trained differentiable models that integrate natural logic with neural networks, aiming to keep the backbone of natural language reasoning based on the natural logic formalism while introducing subsymbolic vector representations and neural components. The proposed model adapts module networks to model natural logic operations, which is enhanced with a memory component to model contextual information. Experiments show that the proposed framework can effectively model monotonicity-based reasoning, compared to the baseline neural network models without built-in inductive bias for monotonicity-based reasoning. Our proposed model shows to be robust when transferred from upward to downward inference. We perform further analyses on the performance of the proposed model on aggregation, showing the effectiveness of the proposed subcomponents on helping achieve better intermediate aggregation performance.

Neuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference

“…We conducted experiments on six datasets: SNLI (Bowman et al, 2015), HELP (Yanaka et al, 2019b), MED (Yanaka et al, 2019a), MoNLI (Geiger et al, 2020), NatLog-2hop (Feng et al, 2020), and a compositional generalization dataset (Yanaka et al, 2020). The results show the model's superior capability in monotonicity inferences, systematic generalization, and interpretability, compared to previous models on these existing datasets, while the model remains a competitive performance on the generic SNLI test set.…”

“…Specifically for natural language, natural logic has long been studied to model reasoning in human language (Lakoff, 1970;van Benthem, 1988;Valencia, 1991;Van Benthem, 1995;Nairn et al, 2006;MacCartney, 2009;MacCartney and Manning, 2009;Icard, 2012;Angeli and Manning, 2014). However, the work of investigating the joint advantage of neural networks and natural logic is sparse (Feng et al, 2020) (See Sec. 2 for more details) and understudied.…”

Automated fact‐checking: A survey

“…Another study explored a symbolic intermediate representation for neural surface realisation (Elder et al, 2019), that is similar to first-order logic. Moreover, a recent attempt adapted module networks to model natural logic operations, which is enhanced with a memory component to model contextual information (Feng et al, 2020). Furthermore, RuleNN (Sen et al, 2020) is developed to tackle sentence classification where models are in the form of first-order logic, and achieved performance, that is comparable to some neural models.…”

Automated Fact-Checking: A Survey

“…Another study explored a symbolic intermediate representation for neural surface realisation [Elder et al, 2019] that is similar to first-order logic. Moreover, a recent attempt adapted module networks to model natural logic operations, which is enhanced with a memory component to model contextual information [Feng et al, 2020]. Furthermore, Ru-leNN [Sen et al, 2020] is developed to tackle sentence classification where models are in the form of first-order logic, and achieved performance that is comparable to some neural models.…”