A Test Collection for Interactive Lifelog Retrieval

Abstract

There is a long history of repeatable and comparable evaluation in Information Retrieval (IR). However, thus far, no shared test collection exists that has been designed to support interactive lifelog retrieval. In this paper we introduce the LSC2018 collection, that is designed to evaluate the performance of interactive retrieval systems. We describe the features of the dataset and we report on the outcome of the first Lifelog Search Challenge (LSC), which used the dataset in an interactive competition at ACM ICMR 2018.

Interactive Video Retrieval in the Age of Deep Learning

“…This part will also discuss the datasets [5,12] currently used for these campaigns. It will discuss the selection and preparation of large-scale datasets and methods of generating ground truth.…”

A Baseline Interactive Retrieval Engine for Visual Lifelogs at the NTCIR-14 Lifelog-3 Task

“…More recently, we note the introduction of a new challenge, specifically aimed at comparing approaches to interactive retrieval from lifelog archives. The Lifelog Search Challenge (LSC) [6] utilises a similar dataset [5] to the one used for the NTCIR14-Lifelog task. The LSC has occurred in 2018 and 2019 and attracted significant interest from participants.…”

“…These changes were combined with a slightly revised interface to take in to account the richer metadata and the content similarity functionality, as shown in Figures 4,5,and 6. In the interactive search competition at LSC2019, this system performed among the top-ranked teams with an overall score of 68, compared to the vitrivr system [19] which was given a score of 100. Interestingly the system significantly closed the gap to the NTCIR-14 system from HCMUS (which also competed at the LSC in 2019) who scored 72 in the competition.…”

Task Intelligence Workshop @ WSDM 2019

“…Associated with the Task Intelligence workshop was a data challenge, for which interested participants could download a rich personal data archive of lifelog data annotated with real-world tasks/activities. For the data challenge, the dataset employed for the Lifelog Search Challenge (LSC) [2], a participating workshop at the 2018 ACM International Conference on Multimedia Retrieval (ICMR) was used. The dataset consisted of 27 days of rich multimodal lifelog data from one lifelogger organized into units of a minute duration.…”