CLEF 2017: Multimodal Spatial Role Labeling (mSpRL) Task Overview

Abstract

The extraction of spatial semantics is important in many real-world applications such as geographical information systems, robotics and navigation, semantic search, etc. Moreover, spatial semantics are the most relevant semantics related to the visualization of language. The goal of multimodal spatial role labeling task is to extract spatial information from free text while exploiting accompanying images. This task is a multimodal extension of spatial role labeling task which has been previously introduced as a semantic evaluation task in the SemEval series. The multimodal aspect of the task makes it appropriate for the CLEF lab series. In this paper, we provide an overview of the task of multimodal spatial role labeling. We describe the task, sub-tasks, corpora, annotations, evaluation metrics, and the results of the baseline and the task participant.

Visually Guided Spatial Relation Extraction from Text

“…It contains 613 images with descriptions including 1, 213 sentences. The standard split of the dataset contains 761 training and 939 testing spatial relations (Kordjamshidi et al, 2017b to this dataset to align phrases in the text with the segments of the related images using brat tool. 4 The alignments are used only for evaluations and are publicly available.…”

Exploring the Functional and Geometric Bias of Spatial Relations Using Neural Language Models

“…The size of this extracted dataset is 96,749 instances and is relatively small for training a neural language model. (Kordjamshidi et al, 2017) released CLEF 2017 multimodal spatial role labelling dataset (mSpRL) which is a human annotated subset of the IAPR TC-12 Benchmark corpus for spatial relations, targets and landmarks (Kordjamshidi et al, 2011) containing 613 text files and 1,213 sentences. While this dataset could not be used to train a language model directly, a spatial role labelling classifier could be trained on it to identify spatial relations and arguments which would then be used to produce a bootstrapped dataset for training a neural language model.…”

Proceedings of the First International Workshop on Spatial Language Understanding

“…The goal is to improve the extraction of spatial information from text by incorporating anaphora resolution for landmark candidates. We briefly define the spatial role labeling (SpRL) task which is based on a previous formalization of ( Kordjamshidi et al, 2017bKordjamshidi et al, , 2011Kordjamshidi and Moens, 2015b). Given a sentence S, segmented into phrases P = [P 1 , P 2 , P 3 , ...P n ] where P i is the identifier of i th phrase in the sentence, the goal of spatial role labeling is to find the phrases which carry spatial roles (i.e.…”