Visual question answering: a state-of-the-art review

Manmadhan, Sruthy; Kovoor, Binsu C.

doi:10.1007/s10462-020-09832-7

Cited by 36 publications

(11 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In deep learning, a question answering system is represented by a more or less complex architecture of neural networks taking at the input a textual question in natu-ral language and a text document [23] or an image [21] [22]. The output should match the correct answer if it exists in the document or image provided.…”

Section: Question Answering Based On Deep Learningmentioning

confidence: 99%

Intelligent System Using Deep Learning for Answering Learner Questions in a MOOC

Hamal

Faddouli

2022

Int. J. Emerg. Technol. Learn.

View full text Add to dashboard Cite

Despite the great success of Massive Open Online Courses (MOOCs), the success rate of learners remains very low. Therefore, a lot of research has been done to understand and solve this abandonment problem. This work is part of the same effort which aims to improve learning in MOOCs. We offer an intelligent system capable of assisting the learner by providing answers to all his questions on the subjects covered in the MOOC. The architecture of our system is based on new advances in artificial intelligence, in particular the applications of deep learning in the field of natural language processing (NLP). The results obtained are quite interesting and demonstrate the relevance of our solution.

show abstract

Section: Question Answering Based On Deep Learningmentioning

confidence: 99%

Intelligent System Using Deep Learning for Answering Learner Questions in a MOOC

Hamal

Faddouli

2022

Int. J. Emerg. Technol. Learn.

View full text Add to dashboard Cite

show abstract

“…All of them address the problem of reasoning about abstract concepts present in the image or identifying hidden rules (also referred to as patterns) that govern visual entities, although in various settings. First of all, such tasks often emerge in the field of Visual Question Answering (VQA) [22][23][24][25][26], where the goal is to answer a question written in natural language referring to an associated image. In VQA, the information present in the image is sufficient for answering the related questions, whereas the Visual Commonsense Reasoning (VCR) field [27][28][29][30][31] takes it a step further and places the tasks in real-world settings, where external knowledge is often required to solve them.…”

Section: Scopementioning

confidence: 99%

“…With the increased number of available AVR benchmarks, more and more methods are proposed to tackle them. Even though in many cases the methods operate on similar inputs and outputs, most of the time they are evaluated only on a single chosen task, without considering the Visual Reasoning (VR) Abstract Visual Reasoning (AVR) ( [17][18][19][20][21] and this work) Visual Question Answering (VQA) [22][23][24][25][26] Visual Commonsense Reasoning (VCR) [27][28][29][30][31] Physical Reasoning (PR) [32][33][34][35][36] Fig. 2.…”

Section: Introductionmentioning

confidence: 99%

A Review of Emerging Research Directions in Abstract Visual Reasoning

Małkiński¹,

Mańdziuk²

2022

Preprint

View full text Add to dashboard Cite

Visual Reasoning (AVR) problems are commonly used to approximate human intelligence. They test the ability of applying previously gained knowledge, experience and skills in a completely new setting, which makes them particularly well-suited for this task. Recently, the AVR problems have become popular as a proxy to study machine intelligence, which has led to emergence of new distinct types of problems and multiple benchmark sets. In this work we review this emerging AVR research and propose a taxonomy to categorise the AVR tasks along 5 dimensions: input shapes, hidden rules, target task, cognitive function, and main challenge. The perspective taken in this survey allows to characterise AVR problems with respect to their shared and distinct properties, provides a unified view on the existing approaches for solving AVR tasks, shows how the AVR problems relate to practical applications, and outlines promising directions for future work. One of them refers to the observation that in the machine learning literature different tasks are considered in isolation, which is in the stark contrast with the way the AVR tasks are used to measure human intelligence, where multiple types of problems are combined within a single IQ test.

show abstract

“…This pioneering work was immediately followed by a vigorous worldwide effort aimed at building new datasets and models (Antol et al., 2015; Gao et al., 2015; Geman et al., 2015; Goyal et al., 2016, 2017; Malinowski et al., 2015; M. Ren, Kiros et al., 2015; Yu et al., 2015). This effort has been exhaustively summarized in various surveys (Kafle & Kanan, 2017b; Manmadhan & Kovoor, 2020; Srivastava et al., 2021; Wu et al., 2017), as well as tutorials (Kordjamshidi et al., 2020; Teney et al., 2017). 1 In particular, Srivastava et al.…”

Section: The Recent Revival Of Vqamentioning

confidence: 99%

“…Ren, Kiros et al, 2015;Yu et al, 2015). This effort has been exhaustively summarized in various surveys (Kafle & Kanan, 2017b;Manmadhan & Kovoor, 2020;Srivastava et al, 2021;Wu et al, 2017), as well as tutorials (Kordjamshidi et al, 2020;Teney et al, 2017). 1 In particular, Srivastava et al ( 2021) nicely sketch the timeline of the major breakthroughs in VQA in the last five years, whilst Wu et al (2017) provide interesting connections with structured knowledge base and an in-depth description of the question/answer pairs present in VQA datasets.…”

Section: The Recent Revival Of Vqamentioning

confidence: 99%

Linguistic issues behind visual question answering

Bernardi

Pezzelle

2021

Language and Linguist. Compass

View full text Add to dashboard Cite

Answering a question that is grounded in an image is a crucial ability that requires understanding the question, the visual context, and their interaction at many linguistic levels: among others, semantics, syntax and pragmatics. As such, visually‐grounded questions have long been of interest to theoretical linguists and cognitive scientists. Moreover, they have inspired the first attempts to computationally model natural language understanding, where pioneering systems were faced with the highly challenging task—still unsolved—of jointly dealing with syntax, semantics and inference whilst understanding a visual context. Boosted by impressive advancements in machine learning, the task of answering visually‐grounded questions has experienced a renewed interest in recent years, to the point of becoming a research sub‐field at the intersection of computational linguistics and computer vision. In this paper, we review current approaches to the problem which encompass the development of datasets, models and frameworks. We conduct our investigation from the perspective of the theoretical linguists; we extract from pioneering computational linguistic work a list of desiderata that we use to review current computational achievements. We acknowledge that impressive progress has been made to reconcile the engineering with the theoretical view. At the same time, we claim that further research is needed to get to a unified approach which jointly encompasses all the underlying linguistic problems. We conclude the paper by sharing our own desiderata for the future.

show abstract

Visual question answering: a state-of-the-art review

Cited by 36 publications

References 77 publications

Intelligent System Using Deep Learning for Answering Learner Questions in a MOOC

Intelligent System Using Deep Learning for Answering Learner Questions in a MOOC

A Review of Emerging Research Directions in Abstract Visual Reasoning

Linguistic issues behind visual question answering

Contact Info

Product

Resources

About