RAVEN: A Dataset for Relational and Analogical Visual REasoNing

Abstract

Dramatic progress has been witnessed in basic vision tasks involving low-level perception, such as object recognition, detection, and tracking. Unfortunately, there is still an enormous performance gap between artificial vision systems and human intelligence in terms of higher-level vision problems, especially ones involving reasoning. Earlier attempts in equipping machines with high-level reasoning have hovered around Visual Question Answering (VQA), one typical task associating vision and language understanding. In this work, we propose a new dataset, built in the context of Raven's Progressive Matrices (RPM) and aimed at lifting machine intelligence by associating vision with structural, relational, and analogical reasoning in a hierarchical representation. Unlike previous works in measuring abstract reasoning using RPM, we establish a semantic link between vision and reasoning by providing structure representation. This addition enables a new type of abstract reasoning by jointly operating on the structure representation. Machine reasoning ability using modern computer vision is evaluated in this newly proposed dataset. Additionally, we also provide human performance as a reference. Finally, we show consistent improvement across all models by incorporating a simple neural module that combines visual understanding and structure reasoning.

Artificial General Intelligence: A New Perspective, with Application to Scientific Discovery

“…Su et al [13] showed that modifying one pixel only could lead up to 73% adversarial success rate depending on the used images. Recently, there is a growing interest in building neural networks that can learn to reason [76][77][78][79]. Saxon et al [77] demonstrated that current state-of-the-art neural networks show moderate performance in solving basic mathematical problems, the performance deteriorates for questions that require the computation of intermediate values.…”

“…The model was able to solve only 14/40 questions from maths exams for 16 year old schoolchildren in the UK. In [78][79] the researchers tested neural networks ability in structural, relational, and analogical reasoning by trying to solve IQ-like visual questions. In particular, they tested the models on the Raven's Progressive Matrices (RPM) dataset, which is correlated with many aspects of reasoning.…”

A Review on Explainability in Multimodal Deep Neural Nets

“…Recently efforts in this direction are started. In [172], a new dataset is proposed based on Raven's Progressive Matrices (RPM) for the task of visual recognition reasoning, comprising images and related RPM problems, with tree-structured annotations. A counting-based dataset is sampled from the available VQA 2.0 and Visual Genome datasets for the task-specific release [200].…”

AI, visual imagery, and a case study on the challenges posed by human intelligence tests

“…More recently, a wave of "data-driven Raven's agents" aims to learn integrated representations of visuospatial domain knowl-edge and problem-solving strategies by training on input-output pairs from a large number of example problems (44)(45)(46)(47)(48)(49).…”