Learning by Fixing: Solving Math Word Problems with Weak Supervision

Abstract

Previous neural solvers of math word problems (MWPs) are learned with full supervision and fail to generate diverse solutions. In this paper, we address this issue by introducing a weakly-supervised paradigm for learning MWPs. Our method only requires the annotations of the final answers and can generate various solutions for a single problem. To boost weakly-supervised learning, we propose a novel learning-by-fixing (LBF) framework, which corrects the misperceptions of the neural network via symbolic reasoning. Specifically, for an incorrect solution tree generated by the neural network, the fixing mechanism propagates the error from the root node to the leaf nodes and infers the most probable fix that can be executed to get the desired answer. To generate more diverse solutions, tree regularization is applied to guide the efficient shrinkage and exploration of the solution space, and a memory buffer is designed to track and save the discovered various fixes for each problem. Experimental results on the Math23K dataset show the proposed LBF framework significantly outperforms reinforcement learning baselines in weakly-supervised learning. Furthermore, it achieves comparable top-1 and much better top-3/5 answer accuracies than fully-supervised methods, demonstrating its strength in producing diverse solutions.

ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills

“…Past works have conducted offline evaluation of GPT-3 [18], the predecessor to the LLM ChatGPT is based on, in computer science education to automatically generate code and error explanations [19][20][21]. GPT-3 has also been applied to math word problems and evaluated on its ability to generate variations of a word problem [22]. Below, we present a literature review of work using other methods to automatically generate hints, provide additional background on LLMs, and contextualize the use of ChatGPT in education.…”

Answering Numerical Reasoning Questions in Table-Text Hybrid Contents with Graph-based Encoder and Tree-based Decoder

“…M op and M con are two trainable embeddings for operators and constants, respectively. For a numeric value in V num , its token embedding takes the corresponding hidden state h i loc(yt,T,P ) , where loc(y t , T, P ) is the index position of y in table T or paragraph P (Hong et al, 2021).…”

SMART: A Situation Model for Algebra Story Problems via Attributed Grammar

“…Researchers recently focus on solving math word problems using neural networks (Ling et al 2017;Wang, Liu, and Shi 2017a;Huang et al 2018a;Robaidek, Koncel-Kedziorski, and Hajishirzi 2018;Wang et al 2018Wang et al , 2019Chiang and Chen 2019;Xie and Sun 2019;Zhang et al 2020;Hong et al 2021). The mere translation from a text to an equation neglects the intermediate process required by problem solving, thus lacking interpretability.…”

“…All rights reserved. from the community of artificial intelligence and natural language processing often mix them with other types of problems, such as number problems and geometry problems, into one whole task called Math Word Problems (MWPs) (Wang, Liu, and Shi 2017a;Huang et al 2016;Amini et al 2019) Recent works on Math Word Problems (Wang, Liu, and Shi 2017a;Huang et al 2018a;Wang et al 2018;Xie and Sun 2019;Hong et al 2021) focused on using end-to-end neural networks (e.g., Seq2Seq, Seq2Tree) to directly translate a problem text into an expression, which is then executed to get the final answer. Although they seem to obtain satisfying performance, such end-to-end neural models suffer from the following drawbacks:…”