Could you give me a hint ? Generating inference graphs for defeasible reasoning

Abstract

Defeasible reasoning is a mode of reasoning where conclusions can be overturned by taking into account new evidence. A commonly used method in cognitive science and logic literature is to handcraft argumentation supporting inference graphs. While humans find inference graphs very useful for reasoning, constructing them at scale is difficult. In this paper, we automatically generate such inference graphs through transfer learning from a related NLP task that shares the kind of reasoning that inference graphs support. Through automated metrics and human evaluation, we find that our method generates meaningful graphs for the defeasible inference task. Human accuracy on this task improves by 20% by consulting the generated graphs. Our findings open up exciting new research avenues for cases where machine reasoning can help human reasoning. 1

Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey

“…By interspersing the inference rules, the storyline generations should create a coherent story that follows logical connections and causal relationships between events. Madaan et al (2021a) employ T5 to generate inference graphs for defeasible inference (Rudinger et al, 2020). In this mode of reasoning, given a premise, a hypothesis may be weakened or overturned in light of new evidence.…”

Think about it! Improving defeasible reasoning by first modeling the question scenario

“…Inspired by past results (Madaan et al, 2021) that humans found inference graphs useful for defeasible inference, we investigate whether neural models can benefit from envisioning the question scenario using an inference graph before answering a defeasible inference query.…”

“…Inference graphs were introduced in philosophy by Pollock (2009) to aid defeasible reasoning for humans, and in NLP by Tandon et al (2019) for a counterfactual reasoning task. We interpret the inference graphs as having four kinds of nodes (Pollock, 2009;Madaan et al, 2021):…”

“…We borrow ideas from the cognitive science literature that supports defeasible reasoning for humans with an inference graph (Pollock, 2009(Pollock, , 1987. Inference graphs formulation in (Madaan et al, 2021), which we use in this paper, draws connections between the P, H, and S through mediating 1 Code and data located at https://github.com/ madaan/thinkaboutit events. This can be seen as a mental model of the question scenario before answering the question (Johnson-Laird, 1983).…”

“…To generate inference graphs, we build upon past work that uses a sequence to sequence approach (Madaan et al, 2021). However, our analysis re-Figure 2: An overview of CURIOUS vealed that the graphs can often be erroneous, and CURIOUS also includes an error correction module to generate higher quality inference graphs.…”