The Most Uncreative Examinee: A First Step toward Wide Coverage Natural Language Math Problem Solving

Matsuzaki, Takashi; Iwane, Hidenao; Anai, Hirokazu; Arai, Noriko

doi:10.1609/aaai.v28i1.8869

Cited by 12 publications

(3 citation statements)

References 24 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, there have been attempts at building systems to pass university entrance exams. Under the National Institute of Informatics' Todai project, several systems were developed for parts of the University of Tokyo Entrance Exam, including mathematics, physics, English, and history (Strickland 2013;Tainaka 2013;Fujita et al 2014), although in some cases questions were modified or annotated before being given to the systems (for example, Matsuzaki et al 2014). Similarly, a smaller project worked on passing the Gaokao (China's college entrance exam; Cheng et al 2016;Guo et al 2017).…”

Section: Related Work On Standardized Testing For Aimentioning

confidence: 99%

From F to A on the New York Regents Science Exams — An Overview of the Aristo Project

et al. 2020

View full text Add to dashboard Cite

AI has achieved remarkable mastery over games such as Chess, Go, and Poker, and even Jeopardy!, but the rich variety of standardized exams has remained a landmark challenge. Even as recently as 2016, the best AI system could achieve merely 59.3 percent on an 8th grade science exam. This article reports success on the Grade 8 New York Regents Science Exam, where for the first time a system scores more than 90 percent on the exam’s nondiagram, multiple choice (NDMC) questions. In addition, our Aristo system, building upon the success of recent language models, exceeded 83 percent on the corresponding Grade 12 Science Exam NDMC questions. The results, on unseen test questions, are robust across different test years and different variations of this kind of test. They demonstrate that modern natural language processing methods can result in mastery on this task. While not a full solution to general question-answering (the questions are limited to 8th grade multiple-choice science) it represents a significant milestone for the field.

show abstract

Section: Related Work On Standardized Testing For Aimentioning

confidence: 99%

From F to A on the New York Regents Science Exams — An Overview of the Aristo Project

et al. 2020

View full text Add to dashboard Cite

show abstract

“…This work is a part of the development of the Todai Robot Math Problem Solver (henceforth ToroboMath) [13][14][15][16]. Figure 2 presents an overview of the system.…”

Section: Todai Robot Math Solver and Problem Librarymentioning

confidence: 99%

Formula Simplification via Invariance Detection by Algebraically Indexed Types

Matsuzaki

Fujita

2022

Automated Reasoning

Self Cite

View full text Add to dashboard Cite

We describe a system that detects an invariance in a logical formula expressing a math problem and simplifies it by eliminating variables utilizing the invariance. Pre-defined function and predicate symbols in the problem representation language are associated with algebraically indexed types, which signify their invariance property. A Hindley-Milner style type reconstruction algorithm is derived for detecting the invariance of a problem. In the experiment, the invariance-based formula simplification significantly enhanced the performance of a problem solver based on quantifier-elimination for real-closed fields, especially on the problems taken from the International Mathematical Olympiads.

show abstract

“…This model should also be able to capture patterns independently of how new or unusual the notations are present in the literature. In 2014, Matsuzaki et al (2014) presented some promising results to answer mathematical questions Japanese university entrance exams automatically. While the approach involves many manual adjustments and analysis, the promising results illustrate the different levels of knowledge that is still required for understanding arXiv documents vs. university entrance level exams.…”

Section: Overcoming Issues Of Knowledge Extractionmentioning

confidence: 99%

Math-word embedding in math search and semantic extraction

et al. 2020

View full text Add to dashboard Cite

Word embedding, which represents individual words with semantically fixed-length vectors, has made it possible to successfully apply deep learning to natural language processing tasks such as semantic role-modeling, question answering, and machine translation. As math text consists of natural text, as well as math expressions that similarly exhibit linear correlation and contextual characteristics, word embedding techniques can also be applied to math documents. However, while mathematics is a precise and accurate science, it is usually expressed through imprecise and less accurate descriptions, contributing to the relative dearth of machine learning applications for information retrieval in this domain. Generally, mathematical documents communicate their knowledge with an ambiguous, context-dependent, and non-formal language. Given recent advances in word embedding, it is worthwhile to explore their use and effectiveness in math information retrieval tasks, such as math language processing and semantic knowledge extraction. In this paper, we explore math embedding by testing it on several different scenarios, namely, (1) math-term similarity, (2) analogy, (3) numerical concept-modeling based on the centroid of the keywords that characterize a concept, (4) math search using query expansions, and (5) semantic extraction, i.e., extracting descriptive phrases for math expressions. Due to the lack of benchmarks, our investigations were performed using the arXiv collection of STEM documents and carefully selected illustrations on the Digital Library of Mathematical Functions (DLMF: NIST digital library of mathematical functions. Release 1.0.20 of 2018-09-1, 2018). Our results show that math embedding holds much promise for similarity, analogy, and search tasks. However, we also observed the need for more robust math embedding approaches. Moreover, we explore and discuss fundamental issues that we believe thwart the progress in mathematical information retrieval in the direction of machine learning.

show abstract

The Most Uncreative Examinee: A First Step toward Wide Coverage Natural Language Math Problem Solving

Cited by 12 publications

References 24 publications

From F to A on the New York Regents Science Exams — An Overview of the Aristo Project

From F to A on the New York Regents Science Exams — An Overview of the Aristo Project

Formula Simplification via Invariance Detection by Algebraically Indexed Types

Math-word embedding in math search and semantic extraction

Contact Info

Product

Resources

About