LifeSeeker 2.0: Interactive Lifelog Search Engine at LSC 2020

Abstract

In this paper we present our interactive lifelog retrieval engine in the LSC'20 comparative benchmarking challenge. The LifeSeeker 2.0 interactive lifelog retrieval engine is developed by both Dublin City University and Ho Chi Minh University of Science, which represents an enhanced version of the two corresponding interactive lifelog retrieval engines in LSC'19. The implementation of LifeSeeker 2.0 has been designed to focus on the searching by text query using a Bag-of-Words model with visual concept augmentation and additional improvements in query processing time, enhanced result display and browsing support, and interacting with visual graphs for both query and filter purposes. CCS CONCEPTS • Information systems → Multimedia databases; Users and interactive retrieval; Search interfaces; • Human-centered computing → Interactive systems and tools.

A Baseline Interactive Retrieval Engine for Visual Lifelogs at the NTCIR-14 Lifelog-3 Task

“…The system was evaluated by four users and findings indicate that the system can be effectively used to locate relevant content. User studies showed that the users generally liked the system, but both observation and feedback provided a list of proposed enhancements to the system, which have been integrated into a new interactive retrieval system called LifeSeeker [9] which was shown to be among the best performers at the LSC2019.…”

“…Through feedback and observation of the users using the retrieval system, we gathered findings that are being used to improve the current system for the LSC'19 (Lifelog Search Challenge) comparative benchmarking exercise. The new system called LifeSeeker [9] is an evolution of this system that incorporated the following updates:…”

Graph-Based Indexing and Retrieval of Lifelog Data

“…With different considerations, Myscéal [28] viewed this as a document retrieval problem by indexing textual annotations and matching with textual queries. Embedding techniques are also commonly based on the idea of encoding concepts from both queries and images tags into the same vector space to calculate the similarity between them [19,20]. Regarding using graphs, LifeGraph [25] applied knowledge graph structure with the nodes representing detected things or scenes recognized in images.…”

Introduction to the Third Annual Lifelog Search Challenge (LSC'20)

“…Che et al [2] integrate relationship-mapping between objects and subjects in the lifelog images. LifeSeeker 2.0 is an enhancement to the 2019 tool of the same name [12], supporting interactive graph-based filtering. The FIRST system [20] is designed to be a flexible integration platform for easy use of new query workflows and visualization layouts.…”