Overview of ImageCLEF 2017: Information Extraction from Images

Abstract

Abstract. This paper presents an overview of the ImageCLEF 2017 evaluation campaign, an event that was organized as part of the CLEF (Conference and Labs of the Evaluation Forum) labs 2017. ImageCLEF is an ongoing initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval for providing information access to collections of images in various usage scenarios and domains. In 2017, the 15th edition of ImageCLEF, three main tasks were proposed and one pilot task: 1) a LifeLog task about searching in LifeLog data, so videos, images and other sources; 2) a caption prediction task that aims at predicting the caption of a figure from the biomedical literature based on the figure alone; 3) a tuberculosis task that aims at detecting the tuberculosis type from CT (Computed Tomography) volumes of the lung and also the drug resistance of the tuberculosis; and 4) a remote sensing pilot task that aims at predicting population density based on satellite images. The strong participation of over 150 research groups registering for the four tasks and 27 groups submitting results shows the interest in this benchmarking campaign despite the fact that all four tasks were new and had to create their own community.

A Baseline Search Engine for Personal Life Archives

“…As a preliminary step, we apply our preliminary version [15] of the proposed baseline search engine to solve the problems in ImageCLEFlifelog 2017 [3] of ImageCLEF 2017 [10]. ImageCLEFlifelog 2017 gives some basic queries on lifelogs retrieval and asks the participant to return the relevant moments.…”

Overview of ImageCLEF 2018: Challenges, Datasets and Evaluation

“…ImageCLEF 2018 consisted of three main tasks and a pilot task that covered challenges in diverse fields and usage scenarios. In 2017 [21] the proposed challenges were almost all new in comparison to 2016 [40], the only exception being Caption Prediction that was a subtask already attempted in 2016, but for which no participant submitted results. After such a big change, for 2018 the objective was to continue most of the tasks from 2017.…”

“…Now, it is getting easier to access data collections but it is still hard to obtain annotated data with a clear evaluation scenario and strong baselines to compare against. Motivated by this, ImageCLEF has for 16 years been an initiative that aims at evaluating multilingual or language independent annotation and retrieval of images [5,21,23,25,39]. The main goal of ImageCLEF is to support the advancement of the field of visual media analysis, classification, annotation, indexing and retrieval.…”

LSE2018 Panel - Challenges of Lifelog Search and Access

“…Thus, there is a strong need to have a common published dataset for studies in this field [6]. To the best of our knowledge, there are very few published dataset in this field (for example the NTCIR-12 [5] and its related version in ImageCLEFlifelog 2017 [7], and there is no common dataset of such kind of data released out of the context of a research challenge or task. The design and construction of such a common published lifelog dataset is also not trivial, where there are significant technical challenges to be solved, arising from the gathering, semantic enrichment, and pervasive accessing of these vast personal data archives [5].…”