XQC at the Lifelog Search Challenge 2021: Interactive Learning on a Mobile Device

Knudsen, Emil; Qvortrup, Thomas Holstein; Khan, Omar Shahbaz; Jónsson, Björn Þór

doi:10.1145/3463948.3469063

Cited by 8 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Pho-toCube [33] interactive retrieval system utilised multidimensional space to explore lifelog data. Finally, LifeMon retrieval prototype [7] explored the use of MongoDB document stores for indexing and XQC [23] introduced an interactive learning interface running on mobile devices.…”

Section: Related Researchmentioning

confidence: 99%

MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2022

Ribiero

Trifan

Neves

2022

Proceedings of the 5th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

Research on retrieving data and analyzing lifelogs revealed to be a very complex task, and the interdisciplinary challenges to be tackled have boosted increasing attention from the scientific community in information retrieval and lifelogging. The Lifelog Search Challenge is an international competition for lifelog retrieval in which researchers propose their approaches and compete to solve lifelog retrieval challenges and evaluate the effectiveness of their systems. In this paper, we present the MEMORIA computational tool to participate for the first time in the Lifelog Search Challenge 2022. The information retrieval is based on the search of keywords and time periods and several computer vision methods are used to process visual lifelogs, from pre-processing algorithms to feature extraction methods, in order to enrich the annotation of the lifelogs. Preliminary experimental results of the user interaction with our retrieval module are presented, confirming the effectiveness of the proposed approach and showing the most relevant functionalities of the system. CCS CONCEPTS• Information systems → Search interfaces; • Human-centered computing → Interactive systems and tools.

show abstract

Section: Related Researchmentioning

confidence: 99%

MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2022

Ribiero

Trifan

Neves

2022

Proceedings of the 5th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

show abstract

“…Lifegraph [28] used the knowledge graph to enhance the annotated information. XQC [18] was one of the first lifelog retrieval systems designed to be used on a mobile device. The system relies on the user's relevance feedback through a number of iterations to gradually find the result, with the addition of a faceted filtering mechanism.…”

Section: Related Workmentioning

confidence: 99%

E-Myscéal: Embedding-based Interactive Lifelog Retrieval System for LSC'22

Tran

Nguyen

et al. 2022

Proceedings of the 5th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

Developing interactive lifelog retrieval systems is a growing research area. There are many international competitions for lifelog retrieval that encourage researchers to build effective systems that can address the multimodal retrieval challenge of lifelogs. The Lifelog Search Challenge (LSC) was first organised in 2018 and is currently the only interactive benchmarking evaluation for lifelog retrieval systems. Participating systems should have an accurate search engine and a user-friendly interface that can help users to retrieve relevant content. In this paper, we upgrade our previous Myscéal, which was the top performing system in LSC'20 and LSC'21, and present E-Myscéal for LSC'22, which includes a completely different search engine. Instead of using visual concepts for retrieval such as Myscéal, the new E-Myscéal employs an embedding technique that facilitates novice users who are not familiar with the concepts. Our experiments show that the new search engine can find relevant images in the first place in the ranked list, four a quarter of the LSC'21 queries (26%) by using just the first hint from the textual information need. Regarding the user interface, we still keep the simple non-faceted design as in the previous version but improve the event view browsing in order to better support novice users. CCS CONCEPTS• Information systems → Information retrieval; • Humancentered computing → Human computer interaction (HCI); User interface design.

show abstract

“…They further added support to train multiple classifiers, results of which could be combined later to support queries that look for events with temporal relations. XQC [16] used the same backend as Exquisitor and proposed a mobile friendly interface to query lifelogs. SOMHunter [17] also leveraged user feedback which re-scores the images using a Bayesian style update.…”

Section: Oral Paper Sessionmentioning

confidence: 99%

Memento 2.0: An Improved Lifelog Search Engine for LSC'22

Alam

Graham

Gurrin

2022

Proceedings of the 5th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

In this paper, we present Memento 2.0, an improved version of our system which first participated in the Lifelog Search Challenge 2021. Memento 2.0 employs image-text embeddings derived from two CLIP models (ViT-L/14 and ResNet-50x64) and adopts a weighted ensemble approach to derive a combined final ranking. Our approach significantly improves the performance over the baseline LSC'21 system. We additionally make important updates to the system's user interface after analysing the shortcomings to make it more efficient and better suited to the needs of the Lifelog Search Challenge. CCS CONCEPTS• Information systems → Retrieval models and ranking; Search interfaces.

show abstract

XQC at the Lifelog Search Challenge 2021: Interactive Learning on a Mobile Device

Cited by 8 publications

References 7 publications

MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2022

MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2022

E-Myscéal: Embedding-based Interactive Lifelog Retrieval System for LSC'22

Memento 2.0: An Improved Lifelog Search Engine for LSC'22

Contact Info

Product

Resources

About