MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2022

Abstract

Research on retrieving data and analyzing lifelogs revealed to be a very complex task, and the interdisciplinary challenges to be tackled have boosted increasing attention from the scientific community in information retrieval and lifelogging. The Lifelog Search Challenge is an international competition for lifelog retrieval in which researchers propose their approaches and compete to solve lifelog retrieval challenges and evaluate the effectiveness of their systems. In this paper, we present the MEMORIA computational tool to participate for the first time in the Lifelog Search Challenge 2022. The information retrieval is based on the search of keywords and time periods and several computer vision methods are used to process visual lifelogs, from pre-processing algorithms to feature extraction methods, in order to enrich the annotation of the lifelogs. Preliminary experimental results of the user interaction with our retrieval module are presented, confirming the effectiveness of the proposed approach and showing the most relevant functionalities of the system. CCS CONCEPTS• Information systems → Search interfaces; • Human-centered computing → Interactive systems and tools.

Blind Image Quality Assessment with Deep Learning: A Replicability Study and Its Reproducibility in Lifelogging

“…By comparing these results to the original model KonCept512, we can conclude that by increasing the batch size to 16, whatever the framework used to train the models, we can achieve the same result as reported by Hosu et al [9]. As our final goal is to integrate a BIQA method into our lifelogging application [13], in order to filter images with low perceptual quality, we ran the original model KonCept512, as well as the model we trained in this reproducibility work, on a lifelogging dataset [18] previously introduced in Section 3.1. We predicted the MOS values of 183,227 images and analyzed the distribution of these images over the possible MOS values as shown in Figure 2.…”

“…So in addition to the models already mentioned, we also trained a model with the implementation in PyTorch following the same procedure mentioned above with a batch size of 8, that fits on our GPU memory. Moreover, in our lifelogging application MEMORIA [13], we implemented all the deep learning modules for image processing in PyTorch. Finally, we tested all the trained models in the test set of the two IQA datasets mentioned in Section 3.1.…”

“…The primary goal of our work is a replicability study of a BIQA deep learning methodology (KonCept512) developed and documented by Hosu et al [9]. Secondly, we are interested in reproducing their work in a lifelogging application [13]. The authors of the study that we replicate in this manuscript proposed a large IQA in-the-wild dataset (KonIQ-10k) (http://database.mmsp-kn.de/koniq-10k-database.html, accessed on 25 November 2022) to evaluate and benchmark IQA algorithms with regard to the authenticity of degradation, the diversity of content, and quality-related indicators.…”

Introduction to the Fifth Annual Lifelog Search Challenge, LSC'22

“…A new system developed for LSC'22 was Memoria [13] which focused on keyword search and a filter-rich interface with enhanced visual annotations incorporating quality selection and various CNNs for richer visual annotation of the visual data. At the time of writing, it is not yet known which of the participants will have the top-performing system.…”

A Deep Learning approach for Transportation Mode Identification using a transformation of Global Positioning System trajectory data into image representations

“…In several self-monitoring applications explored in the literature [2,5], such as lifelogging applications, raw GPS data does not directly provide specific location names or information about the transport mode being used by an individual. This highlights the necessity of utilizing data processing and mining algorithms to extract valuable insights from the collected data [6].…”