Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks

Kordjamshidi, Parisa; Pustejovsky, James; Moens, Marie‐Francine

doi:10.18653/v1/2020.emnlp-tutorials.5

Cited by 5 publications

(4 citation statements)

References 23 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This pioneering work was immediately followed by a vigorous worldwide effort aimed at building new datasets and models (Antol et al., 2015; Gao et al., 2015; Geman et al., 2015; Goyal et al., 2016, 2017; Malinowski et al., 2015; M. Ren, Kiros et al., 2015; Yu et al., 2015). This effort has been exhaustively summarized in various surveys (Kafle & Kanan, 2017b; Manmadhan & Kovoor, 2020; Srivastava et al., 2021; Wu et al., 2017), as well as tutorials (Kordjamshidi et al., 2020; Teney et al., 2017). 1 In particular, Srivastava et al.…”

Section: The Recent Revival Of Vqamentioning

confidence: 99%

“…Ren, Kiros et al, 2015;Yu et al, 2015). This effort has been exhaustively summarized in various surveys (Kafle & Kanan, 2017b;Manmadhan & Kovoor, 2020;Srivastava et al, 2021;Wu et al, 2017), as well as tutorials (Kordjamshidi et al, 2020;Teney et al, 2017). 1 In particular, Srivastava et al ( 2021) nicely sketch the timeline of the major breakthroughs in VQA in the last five years, whilst Wu et al (2017) provide interesting connections with structured knowledge base and an in-depth description of the question/answer pairs present in VQA datasets.…”

Section: The Recent Revival Of Vqamentioning

confidence: 99%

See 1 more Smart Citation

Linguistic issues behind visual question answering

Bernardi

Pezzelle

2021

Language and Linguist. Compass

View full text Add to dashboard Cite

Answering a question that is grounded in an image is a crucial ability that requires understanding the question, the visual context, and their interaction at many linguistic levels: among others, semantics, syntax and pragmatics. As such, visually‐grounded questions have long been of interest to theoretical linguists and cognitive scientists. Moreover, they have inspired the first attempts to computationally model natural language understanding, where pioneering systems were faced with the highly challenging task—still unsolved—of jointly dealing with syntax, semantics and inference whilst understanding a visual context. Boosted by impressive advancements in machine learning, the task of answering visually‐grounded questions has experienced a renewed interest in recent years, to the point of becoming a research sub‐field at the intersection of computational linguistics and computer vision. In this paper, we review current approaches to the problem which encompass the development of datasets, models and frameworks. We conduct our investigation from the perspective of the theoretical linguists; we extract from pioneering computational linguistic work a list of desiderata that we use to review current computational achievements. We acknowledge that impressive progress has been made to reconcile the engineering with the theoretical view. At the same time, we claim that further research is needed to get to a unified approach which jointly encompasses all the underlying linguistic problems. We conclude the paper by sharing our own desiderata for the future.

show abstract

Section: The Recent Revival Of Vqamentioning

confidence: 99%

Section: The Recent Revival Of Vqamentioning

confidence: 99%

Linguistic issues behind visual question answering

Bernardi

Pezzelle

2021

Language and Linguist. Compass

View full text Add to dashboard Cite

show abstract

“…This can be clearly seen in Figure 1c in which "north of California" is misinterpreted as "Northern California" which in turn causes the QA failure. In fact, the difficulty of recognizing spatial relations from natural language sentences has attracted a lot of attention from the NLP and machine learning community (Kordjamshidi et al, 2020), especially in the domain of visual question answering (Antol et al, 2015). Many papers are focusing on recognizing spatial relations which are viewpoint dependent (Ramalho et al, 2018) such as on the left of this door, on the right of this building, behind this desk.…”

Section: Introductionmentioning

confidence: 99%

Geographic Question Answering: Challenges, Uniqueness, Classification, and Future Directions

Mai

Janowicz

Zhu

et al. 2021

AGILE GIScience Ser.

View full text Add to dashboard Cite

Abstract. As an important part of Artificial Intelligence (AI), Question Answering (QA) aims at generating answers to questions phrased in natural language. While there has been substantial progress in open-domain question answering, QA systems are still struggling to answer questions which involve geographic entities or concepts and that require spatial operations. In this paper, we discuss the problem of geographic question answering (GeoQA). We first investigate the reasons why geographic questions are difficult to answer by analyzing challenges of geographic questions. We discuss the uniqueness of geographic questions compared to general QA. Then we review existing work on GeoQA and classify them by the types of questions they can address. Based on this survey, we provide a generic classification framework for geographic questions. Finally, we conclude our work by pointing out unique future research directions for GeoQA.

show abstract

mentioning

confidence: 99%

Geographic Question Answering: Challenges, Uniqueness, Classification, and Future Directions

Mai

Janowicz

Zhu

et al. 2021

Preprint

View full text Add to dashboard Cite

As an important part of Artificial Intelligence (AI), Question Answering (QA) aims at generating answers to questions phrased in natural language. While there has been substantial progress in open-domain question answering, QA systems are still struggling to answer questions which involve geographic entities or concepts and that require spatial operations. In this paper, we discuss the problem of geographic question answering (GeoQA). We first investigate the reasons why geographic questions are difficult to answer by analyzing challenges of geographic questions. We discuss the uniqueness of geographic questions compared to general QA. Then we review existing work on GeoQA and classify them by the types of questions they can address. Based on this survey, we provide a generic classification framework for geographic questions. Finally, we conclude our work by pointing out unique future research directions for GeoQA.

show abstract

Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks

Cited by 5 publications

References 23 publications

Linguistic issues behind visual question answering

Linguistic issues behind visual question answering

Geographic Question Answering: Challenges, Uniqueness, Classification, and Future Directions

Geographic Question Answering: Challenges, Uniqueness, Classification, and Future Directions

Contact Info

Product

Resources

About