STaR: Bootstrapping Reasoning With Reasoning

Zelikman, Eric; Wu, Yuhuai; Goodman, Noah D.

doi:10.48550/arxiv.2203.14465

Cited by 14 publications

(22 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Language models (LMs) have demonstrated impressive few-shot learning abilities (Brown et al, 2020;Chowdhery et al, 2022). This has led to a number of proposals to use LMs as the basis of informal reasoning, including scratchpads (Nye et al, 2021), chain of thought prompting (Wei et al, 2022;Wang et al, 2022), learned verifiers (Cobbe et al, 2021), selection-inference (Creswell et al, 2022), and bootstrapping (Zelikman et al, 2022). They have also been applied in formal mathematics settings to guide theorem provers (Polu & Sutskever, 2020).…”

Section: Introductionmentioning

confidence: 99%

Language Model Cascades

Dohan¹,

Xu²,

Lewkowycz³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. These compositions are probabilistic models, and may be expressed in the language of graphical models with random variables whose values are complex data types such as strings. Cases with control flow and dynamic structure require techniques from probabilistic programming, which allow implementing disparate model structures and inference strategies in a unified language. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use. We refer to the resulting programs as language model cascades.

show abstract

Section: Introductionmentioning

confidence: 99%

Language Model Cascades

Dohan¹,

Xu²,

Lewkowycz³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Fig. 1a); 2) approaches that encourage LLMs to produce reasoning explicitly, but all reasoning steps are produced in one generative step (Cobbe et al, 2021;Dalvi et al, 2021;Jhamtani and Clark, 2020;Nye et al, 2021a;Wei et al, 2022;Zelikman et al, 2022) (e.g. Fig.…”

Section: Related Workmentioning

confidence: 99%

“…However, although explicit reasoning helps improve the accuracy of the models, encouraging the models to produce multiple steps of reasoning in a single generative pass is not enough to make the models use reasoning in a causal manner. The authors found that the generated reasoning traces often contain unrelated or incorrect steps while still resulting in the correct answer (see examples in the appendices of (Wei et al, 2022;Zelikman et al, 2022)). Encouraging LLMs to generate each reasoning step one at a time is currently the most promising direction for achieving causal reasoning, and it is the approach we take in our paper.…”

Section: Related Workmentioning

confidence: 99%

Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning

Creswell¹,

Shanahan²,

Higgins³

2022

Preprint

View full text Add to dashboard Cite

Large language models (LLMs) have been shown to be capable of impressive few-shot generalisation to new tasks. However, they still tend to perform poorly on multi-step logical reasoning problems.Here we carry out a comprehensive evaluation of LLMs on 50 tasks that probe different aspects of logical reasoning. We show that language models tend to perform fairly well at single step inference or entailment tasks, but struggle to chain together multiple reasoning steps to solve more complex problems. In light of this, we propose a Selection-Inference (SI) framework that exploits pre-trained LLMs as general processing modules, and alternates between selection and inference to generate a series of interpretable, casual reasoning steps leading to the final answer. We show that a 7B parameter LLM used within the SI framework in a 5-shot generalisation setting, with no fine-tuning, yields a performance improvement of over 100% compared to an equivalent vanilla baseline on a suite of 10 logical reasoning tasks. The same model in the same setting even outperforms a significantly larger 280B parameter baseline on the same suite of tasks. Moreover, answers produced by the SI framework are accompanied by a causal natural-language-based reasoning trace, which has important implications for the safety and trustworthiness of the system.

show abstract

“…Recent work utilized the ability of large LMs to generate intermediate reasoning steps for improving performance on QA tasks Zelikman et al, 2022;Nye et al, 2022). Specifically, introduced a 'chain-of-thought' prompting, to elicit intermediate reasoning steps along with answers from LMs, which improved performance on several reasoning tasks.…”

Section: Related Workmentioning

confidence: 99%

Inferring Implicit Relations with Language Models

Katz¹,

Geva²,

Berant³

2022

Preprint

View full text Add to dashboard Cite

A prominent challenge for modern language understanding systems is the ability to answer implicit reasoning questions, where the required reasoning steps for answering the question are not mentioned in the text explicitly. In this work, we investigate why current models struggle with implicit reasoning question answering (QA) tasks, by decoupling inference of reasoning steps from their execution. We define a new task of implicit relation inference and construct a benchmark, IMPLICITRE-LATIONS, where given a question, a model should output a list of concept-relation pairs, where the relations describe the implicit reasoning steps required for answering the question. Using IMPLICITRELATIONS, we evaluate models from the GPT-3 family and find that, while these models struggle on the implicit reasoning QA task, they often succeed at inferring implicit relations. This suggests that the bottleneck for answering implicit reasoning questions is in the ability of language models to retrieve and reason over information rather than to plan an accurate reasoning strategy.Recent advances in QA Lourie et al., 2021) have steered attention towards implicit reasoning QA benchmarks such as STRAT-EGYQA (Geva et al., 2021), OPENCSR (Lin et al., 2021), COMMONSENSEQA 2.0 (Talmor et al., 2021, CREAK (Onoe et al., 2021), and REALFP (Kalyan et al., 2021), which span a wide range of domains and reasoning skills. Still, implicit reasoning remains an open challenge, even for large language models (LMs) such as GPT-3 and PaLM (BIG-bench collab.

show abstract

STaR: Bootstrapping Reasoning With Reasoning

Cited by 14 publications

References 13 publications

Language Model Cascades

Language Model Cascades

Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning

Inferring Implicit Relations with Language Models

Contact Info

Product

Resources

About