ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning

Abstract

Recent commonsense-reasoning tasks are typically discriminative in nature, where a model answers a multiple-choice question for a certain context. Discriminative tasks are limiting because they fail to adequately evaluate the model's ability to reason and explain predictions with underlying commonsense knowledge. They also allow such models to use reasoning shortcuts and not be "right for the right reasons". In this work, we present EX-PLAGRAPHS, a new generative and structured commonsense-reasoning task (and an associated dataset) of explanation graph generation for stance prediction. Specifically, given a belief and an argument, a model has to predict

Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning

“…Representative works on graph generation from language models include knowledge graph completion models like Comet Hwang et al, 2021) that fine-tune GPT (Radford et al, 2019;Brown et al, 2020) and BART (Lewis et al, 2020), generation of event influence graphs (Tandon et al, 2019;Madaan et al, 2020), partially ordered scripts (Sakaguchi et al, 2021), temporal graphs (Madaan and Yang, 2021), entailment trees , proof graphs (Saha et al, 2020;Saha et al, 2021a) and commonsense explanation graphs (Saha et al, 2021b). Linguistic tasks like syntactic parsing Mohammadshahi and Henderson, 2021;Kondratyuk and Straka, 2019) and semantic parsing (Chen et al, 2020b;Shin et al, 2021) have also made use of language models.…”

“…Generative Commonsense Reasoning. While traditional commonsense reasoning tasks are discriminative in nature (Zellers et al, 2018;Talmor et al, 2019;Bisk et al, 2020;Sakaguchi et al, 2020;Talmor et al, 2021), recent focus on generative evaluation have led to the development of tasks and benchmarks that explore unstructured commonsense sentence generation (Lin et al, 2020), event influence graph generation (Madaan et al, 2020), commonsense explanation graph generation (Saha et al, 2021b), etc. We experiment with two graph generation tasks, primarily focusing on ExplaGraphs (Saha et al, 2021b) because of the clear distinction in the underlying structural constraints and the semantic aspect dealing with commonsense.…”

“…For example, consider a recently proposed commonsense explanation graph generation task shown in Fig. 1 (Saha et al, 2021b). Each example shows a belief, an argument and an explanation graph explaining how the argument supports or refutes the belief.…”

“…Following Saha et al (2021b), we represent graphs as strings composed of concatenated edges and fine-tune T5 to generate graphs in an autoregressive manner. We observe that while moderate amount of supervision enables the model to learn valid graph encodings, the graphs frequently violate task-specific structural constraints (like connectivity).…”

“…While a general recipe towards improving the structural and semantic aspects of graph generation can be via large-scale training with more humanannotated graphs, it is prohibitive under most practical scenarios because of the cognitive load associated with a complex data creation task like graph annotation Saha et al, 2021b). Hence, we propose simple yet effective methods of graph perturbations that perform various kinds of node and edge addition, deletion, and replacement operations to construct structurally and semantically positive (correct) and negative (incorrect) graphs.…”