Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model

Abstract

Explainable question answering systems should produce not only accurate answers but also rationales that justify their reasoning and allow humans to check their work. But what sorts of rationales are useful and how can we train systems to produce them? We propose a new style of rationale for open-book question answering, called markup-and-mask, which combines aspects of extractive and free-text explanations. In the markup phase, the passage is augmented with free-text markup that enables each sentence to stand on its own outside the discourse context. In the masking phase, a sub-span of the marked-up passage is selected. To train a system to produce markup-and-mask rationales without annotations, we leverage in-context learning. Specifically, we generate silver annotated data by sending a series of prompts to a frozen pretrained language model, which acts as a teacher. We then fine-tune a smaller student model by training on the subset of rationales that led to correct answers. The student is "honest" in the sense that it is a pipeline: the rationale acts as a bottleneck between the passage and the answer, while the "untrusted" teacher operates under no such constraints. Thus, we offer a new way to build trustworthy pipeline systems from a combination of end-task annotations and frozen pretrained language models.

Distilling Reasoning Capabilities into Smaller Language Models

“…Snell et al (2022) demonstrated the usefulness of providing instruction that can help models achieve better reasoning skills. Similar to our hypothesis, Eisenstein et al (2022) argued that question-answering systems should focus not only on the final answer, but also on the rationale that justifies their reasoning, to help them reason better. We go beyond this; in our work, in addition to the question-answering system, we also focus on what questions need to be asked at each step that can help to learn that reasoning step better.…”

AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning

“…Examples of these tasks include math problems [38,88,101,145,158], blurry text recognition [92,93], and matching images with the same content [74,109]. For these tasks, workflows can improve performance on benchmarks [31,42,51,74,87,107,126,145,156], enable new tasks [47,50,58,62,93], classify examples for which machine learning approaches struggle [4,15,25,109], improve instructional clarity [17], and reduce hallucinations [32,33,108,110,111,116,140]. Verifiable creativity.…”

“…One workflow may employ multiple architectural patterns. Sequential architectures order subtasks in a chain in which the output of one worker passes forward to another worker [42,45,56,57,97,110,122,135,142,143,158].…”

“…Most workflows involve at least one step in which workers generate content. This generation may be writing words or code [87,118,138,155,156], answering a question or solving a problem [42,74,145,146], or brainstorming and ideation [29,59,83,97,102,150]. Generation tasks may also ask for labels or annotations on given content [18,23,27,50,51], to translate between languages [6,151], to contribute to a conversation with other workers [38,88,102,134], to demonstrate a behavior [16,85], or to source content from external sources [29,30,53,60,66,67,98,120].…”

“…Other subtasks improve content generated by previous steps, sometimes by incorporating feedback from validation steps. Improving on previously generated content can come in many forms: updating the format of the content [12,42,57,134,142], pruning bad results [45,95], adding missing results [57,95,138], correcting errors [37,47,57,74,148], and performing direct improvements [15,30,35,51,53,62,68,85,89,93,101,102,107,113,126,131,143]. Transform subtasks primarily support the strategies of resolving disagreements among diverse responses.…”