LifeGraph: A Knowledge Graph for Lifelogs

Rossetto, Luca; Baumgartner, Matthias; Ashena, Narges; Ruosch, Florian; Pernisch, Romana; Bernstein, Abraham

doi:10.1145/3379172.3391717

Cited by 16 publications

(13 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several systems have tried to address the issues regarding semantic gap between query and images, and poor contextual understanding of the data. FIRST [25] uses an autoencoder like approach to map query text and images into a common semantic space to measure the similarity between them, LifeGraph [23] used a knowledge graph to represent the lifelog data to capture the internal relations of the various data modalities and linked it to external static data sources for better semantic understanding. Chu et al [6] extracted relation graphs from lifelog images to better describe the relationship between entities (subject-object) present within the image.…”

Section: Related Workmentioning

confidence: 99%

Memento: A Prototype Lifelog Search Engine for LSC'21

Alam

Graham

Gurrin

2021

Proceedings of the 4th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

In this paper, we introduce a new lifelog retrieval system called Memento that leverages semantic representations of images and textual queries projected into a common latent space to facilitate effective retrieval. It bridges the semantic gap between complex visual scenes/events and user information needs expressed as textual and faceted queries. The system, developed for the 2021 Lifelog Search Challenge, also has a minimalist user interface that includes primary search, temporal search, and visual data filtering components. CCS CONCEPTS• Information systems → Information retrieval; Search interfaces; Retrieval models and ranking.

show abstract

Section: Related Workmentioning

confidence: 99%

Memento: A Prototype Lifelog Search Engine for LSC'21

Alam

Graham

Gurrin

2021

Proceedings of the 4th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

show abstract

“…Embedding techniques are also commonly based on the idea of encoding concepts from both queries and images tags into the same vector space to calculate the similarity between them [19,20]. Regarding using graphs, LifeGraph [25] applied knowledge graph structure with the nodes representing detected things or scenes recognized in images. These entities can be linked with corresponding images and external sources to expand the information with frequent activities and relevant objects.…”

Section: Related Workmentioning

confidence: 99%

Graph-Based Indexing and Retrieval of Lifelog Data

Nguyen

Gurrin

2021

MultiMedia Modeling

View full text Add to dashboard Cite

Understanding the relationship between objects in an image is an important challenge because it can help to describe actions in the image. In this paper, a graphical data structure, named "Scene Graph", is utilized to represent an encoded informative visual relationship graph for an image, which we suggest has a wide range of potential applications. This scene graph is applied and tested in the popular domain of lifelogs, and specifically in the challenge of known-item retrieval from lifelogs. In this work, every lifelog image is represented by a scene graph, and at retrieval time, this scene graph is compared with the semantic graph, parsed from a textual query. The result is combined with location or date information to determine the matching items. The experiment shows that this technique can outperform a conventional method.

show abstract

“…This idea was adopted by almost the half of the participants in the LSC'20 [17,20,26,27,39]. On the other hand, SOMHunter [27] which was the runner-up system of the competition and Exquisitor [17] heavily relied on user's relevance feedback, while LifeGraph [33] explored the potential of indexing the lifelog into a graph structure [30].…”

Section: Related Workmentioning

confidence: 99%

Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21

Tran

Nguyen

Bình

et al. 2021

Proceedings of the 4th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

Building an interactive retrieval system for lifelogging contains many challenges due to massive multi-modal personal data besides the requirement of accuracy and rapid response for such a tool. The Lifelog Search Challenge (LSC) is the international lifelog retrieval competition that inspires researchers to develop their systems to cope with the challenges and evaluates the effectiveness of their solutions. In this paper, we upgrade our previous Myscéal and present Myscéal 2.0 system for the LSC'21 with the improved features inspired by the novice users experiments. The experiments show that a novice user achieved more than half of the expert score on average. To mitigate the gap of them, some potential enhancements were identified and integrated to the enhanced version. CCS CONCEPTS• Information systems → Information retrieval; • Humancentered computing → Human computer interaction (HCI); User interface design.

show abstract

LifeGraph: A Knowledge Graph for Lifelogs

Cited by 16 publications

References 15 publications

Memento: A Prototype Lifelog Search Engine for LSC'21

Memento: A Prototype Lifelog Search Engine for LSC'21

Graph-Based Indexing and Retrieval of Lifelog Data

Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21

Contact Info

Product

Resources

About