Enhanced VIRET Tool for Lifelog Data

Nguyen

Proceedings of the Third Annual Workshop on Lifelog Search Challenge

et al. 2020

Lifelog can provide useful insights of our daily activities. It is essential to provide a flexible way for users to retrieve certain events or moments of interest, corresponding to a wide variation of query types. This motivates us to develop FIRST, a Flexible Interactive Retrieval SysTem, to help users to combine or integrate various query components in a flexible manner to handle different query scenarios, such as visual clustering data based on color histogram, visual similarity, GPS location, or scene attributes. We also employ personalized concept detection and image captioning to enhance image understanding from visual lifelog data, and develop an autoencoderlike approach for query text and image feature mapping. Furthermore, we refine the user interface of the retrieval system to better assist users in query expansion and verifying sequential events in a flexible temporal resolution to control the navigation speed through sequences of images. CCS CONCEPTS • Information systems → Search interfaces; Multimedia databases; • Human-centered computing → Interactive systems and tools.

Section: Related Workmentioning

confidence: 99%

FIRST - Flexible Interactive Retrieval SysTem for Visual Lifelog Exploration at LSC 2020

Nguyen

Proceedings of the Third Annual Workshop on Lifelog Search Challenge

et al. 2020

“…The engine was also equipped boolean querying as a late-filtering approach to fuse the results. VIRET [19] is another video retrieval system at VBS which proved to work perfectly on lifelog data by considering day and image of a day as "video" and "shot" correspondingly. In LSC2019, VIRET was upgraded with a query panel to allow better filtering by the metadata (week days, heart rate, time) and the GPS locations to help the system narrow down the images to search for.…”

Section: Related Researchmentioning

confidence: 99%

LifeSeeker 2.0: Interactive Lifelog Search Engine at LSC 2020

Ninh

Proceedings of the Third Annual Workshop on Lifelog Search Challenge

et al. 2020

In this paper we present our interactive lifelog retrieval engine in the LSC'20 comparative benchmarking challenge. The LifeSeeker 2.0 interactive lifelog retrieval engine is developed by both Dublin City University and Ho Chi Minh University of Science, which represents an enhanced version of the two corresponding interactive lifelog retrieval engines in LSC'19. The implementation of LifeSeeker 2.0 has been designed to focus on the searching by text query using a Bag-of-Words model with visual concept augmentation and additional improvements in query processing time, enhanced result display and browsing support, and interacting with visual graphs for both query and filter purposes. CCS CONCEPTS • Information systems → Multimedia databases; Users and interactive retrieval; Search interfaces; • Human-centered computing → Interactive systems and tools.

Proceedings of the Third Annual Workshop on Lifelog Search Challenge

“…In the comparatively short time the Lifelog Search Challenge offers a common basis for the evaluation of retrieval approaches geared towards lifelog data, participating teams have already begun to explore several aspects of the retrieval process. Multiple teams started out adapting their retrieval approaches originally developed for the Video Browser Showdown [25], an interactive video retrieval campaign, to the similarly structured lifelog retrieval task, which lead to a strong focus on the visual aspects of the lifelog data [16,18,22]. Other systems have experimented with novel user interaction approaches.…”

Section: Related Workmentioning

confidence: 99%

LifeGraph: A Knowledge Graph for Lifelogs

Rossetto

Baumgartner

Ashena

et al. 2020

The data produced by efforts such as life logging is commonly multi modal and can have manifold interrelations with itself as well as external information. Representing this data in such a way that these rich relations as well as all the different sources can be leveraged is a non-trivial undertaking. In this paper, we present the first iteration of LifeGraph, a Knowledge Graph for lifelogging data. Life-Graph aims at not only capturing all aspects of the data contained in a lifelog but also linking them to external, static knowledge bases in order to put the log as a whole as well as its individual entries into a broader context. In the Lifelog Search Challenge 2020, we show a first proof-of-concept implementation of LifeGraph as well as a retrieval system prototype which utilizes it to search the log for specific events.