How Well Do Machines Perform on IQ tests: a Comparison Study on a Large-Scale Dataset

Liu, Yusen; He, Feng; Zhang, Haodi; Rao, Guozheng; Feng, Zhiyong; Zhou, Yi

doi:10.24963/ijcai.2019/846

Cited by 9 publications

(9 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Between 2006 and 2011, computational models aimed at solving IQ test problems started to become more popular, either trying to understand human cognition or as a method to evaluate AI techniques [16]. Meanwhile, between 2011 and 2014, several models were proposed to solve IQ tests based on numerical series, although some of them performed (in general) worse than human beings [3].…”

Section: Resolution Of Iq Test and Numerical Se-ries: Previous Modelsmentioning

confidence: 99%

“…In particular, pattern recognition has traditionally been used to assess inductive reasoning skills in IQ tests. It is natural that this type of problem is of interest in AI, where different models have been developed to solve IQ tests [3], [4], [5] and the prediction of numerical sequences has been proposed as a method to evaluate the computational capabilities of machine learning models [6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

KitBit: A New AI Model for Solving Intelligence Tests and Numerical Series

Corsino¹,

Gilpérez²,

Herrera³

2022

Preprint

View full text Add to dashboard Cite

<p>The resolution of intelligence tests, in particular numerical sequences, has been of great interest in the evaluation of AI systems. We present a new computational model called KitBit that uses a reduced set of algorithms and their combinations to build a predictive model that finds the underlying pattern in numerical sequences, such as those included in IQ tests and others of much greater complexity. We present the fundamentals of the model and its application in different cases. First, the system is tested on a set of number series used in IQ tests collected from various sources. Next, our model is successfully applied on the sequences used to evaluate the models reported in the literature. In both cases, the system is capable of solving these types of problems in less than a second using standard computing power. Finally, KitBit's algorithms have been applied for the first time to the complete set of entire sequences of the well-known OEIS database. We find a pattern in the form of a list of algorithms and predict the following terms in the largest number of series to date. These results demonstrate the potential of KitBit to solve complex problems that could be represented numerically.</p>

show abstract

Section: Resolution Of Iq Test and Numerical Se-ries: Previous Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

KitBit: A New AI Model for Solving Intelligence Tests and Numerical Series

Corsino¹,

Gilpérez²,

Herrera³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For BERT + refine (all), we sample over all valid logical forms according to a uniform distribution. For BERT + refine (1,2,5), BERT + refine (2,4,5), and BERT + refine (4,7,9), logical forms are uniformly sampled over (#1, #2, #5), (#2, #4, #5), and (#4 , #7, #9), respectively. The training procedures follow the same hyper-parameters described above.…”

Section: Supplementary Materials a Experimental Detailsmentioning

confidence: 99%

“…Recently, pretrained language models [4,11,19] have achieved great successes on various natural language understanding tasks, and they are also believed to master a certain level of commonsense reasoning abilities [9,10,17]. Equipping machines with commonsense reasoning ability has been seen as one of the key milestones of artificial general intelligence [3].…”

Section: Introductionmentioning

confidence: 99%

Teaching Pretrained Models with Commonsense Reasoning: A Preliminary KB-Based Approach

Chen²,

Yu³

2019

Preprint

View full text Add to dashboard Cite

Recently, pretrained language models (e.g., BERT) have achieved great success on many downstream natural language understanding tasks and exhibit a certain level of commonsense reasoning ability. However, their performance on commonsense tasks is still far from that of humans. As a preliminary attempt, we propose a simple yet effective method to teach pretrained models with commonsense reasoning by leveraging the structured knowledge in ConceptNet, the largest commonsense knowledge base (KB). Specifically, the structured knowledge in KB allows us to construct various logical forms, and then generate multiple-choice questions requiring commonsense logical reasoning. Experimental results demonstrate that, when refined on these training examples, the pretrained models consistently improve their performance on tasks that require commonsense reasoning, especially in the few-shot learning setting. Besides, we also perform analysis to understand which logical relations are more relevant to commonsense reasoning.

show abstract

“…Recently, data sets such as Kandinsky patterns (Mueller & Holzinger, 2019; and CLEVR (Johnson et al, 2017) have been proposed to assess the performance of the machine learning systems in object-centric reasoning tasks. For example, Figure 1 (Bruner et al, 1956;Dowe & Hernández-Orallo, 2012;Liu et al, 2019), which require humans to think on abstract patterns. The key feature of Kandinsky Patterns is its complexity, e.g., the arrangement of objects, closure or symmetry, and a group of objects.…”

Section: Introductionmentioning

confidence: 99%

Neuro-Symbolic Forward Reasoning

Shindo¹,

Dhami²,

Kersting³

2021

Preprint

View full text Add to dashboard Cite

Reasoning is an essential part of human intelligence and thus has been a longstanding goal in artificial intelligence research. With the recent success of deep learning, incorporating reasoning with deep learning systems, i.e., neuro-symbolic AI has become a major field of interest. We propose the Neuro-Symbolic Forward Reasoner (NSFR), a new approach for reasoning tasks taking advantage of differentiable forward-chaining using first-order logic. The key idea is to combine differentiable forward-chaining reasoning with object-centric (deep) learning. Differentiable forward-chaining reasoning computes logical entailments smoothly, i.e., it deduces new facts from given facts and rules in a differentiable manner. The object-centric learning approach factorizes raw inputs into representations in terms of objects. Thus, it allows us to provide a consistent framework to perform the forward-chaining inference from raw inputs. NSFR factorizes the raw inputs into the object-centric representations, converts them into probabilistic ground atoms, and finally performs differentiable forward-chaining inference using weighted rules for inference. Our comprehensive experimental evaluations on object-centric reasoning data sets, 2D Kandinsky patterns and 3D CLEVR-Hans, and a variety of tasks show the effectiveness and advantage of our approach.

show abstract

How Well Do Machines Perform on IQ tests: a Comparison Study on a Large-Scale Dataset

Cited by 9 publications

References 5 publications

KitBit: A New AI Model for Solving Intelligence Tests and Numerical Series

KitBit: A New AI Model for Solving Intelligence Tests and Numerical Series

Teaching Pretrained Models with Commonsense Reasoning: A Preliminary KB-Based Approach

Neuro-Symbolic Forward Reasoning

Contact Info

Product

Resources

About