Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21

Tran, Ly-Duyen; Nguyen, Manh-Duy; Bình, Nguyễn Thanh; Lee, Hyowon; Gurrin, Cathal

doi:10.1145/3463948.3469064

Cited by 30 publications

(10 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most common approach to organizing, retrieving, and analyzing data from wearable cameras involves assigning semantic contexts to images, like visual descriptions, time, and location [ 13 , 14 ]. Various computer vision models are employed to extract visual information from the images, including object detection, activity recognition, optical character recognition [ 13 , 15 ], and embedding models [ 16 , 17 ]. A typical retrieval system would also incorporate different techniques, namely, query enhancement [ 13 ], visual similarity search [ 16 ], and temporal search [ 16 ].…”

Section: Discussionmentioning

confidence: 99%

“…This involves assigning semantic contexts like visual descriptions, time, and location [ 13 , 14 ]. Various computer vision models are employed, such as object detection, activity recognition, and optical character recognition, in addition to embedding models [ 13 , 15 - 17 ]. Retrieval systems incorporate techniques such as query enhancement, visual similarity search, and temporal search [ 13 , 16 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Daily Activity Lifelogs of People With Heart Failure: Observational Study

Tegegne,

Tran,

Nourse

et al. 2024

JMIR Form Res

Self Cite

View full text Add to dashboard Cite

Background Globally, heart failure (HF) affects more than 64 million people, and attempts to reduce its social and economic burden are a public health priority. Interventions to support people with HF to self-manage have been shown to reduce hospitalizations, improve quality of life, and reduce mortality rates. Understanding how people self-manage is imperative to improve future interventions; however, most approaches to date, have used self-report methods to achieve this. Wearable cameras provide a unique tool to understand the lived experiences of people with HF and the daily activities they undertake, which could lead to more effective interventions. However, their potential for understanding chronic conditions such as HF is unclear. Objective This study aimed to determine the potential utility of wearable cameras to better understand the activities of daily living in people living with HF. Methods The “Seeing is Believing (SIB)” study involved 30 patients with HF who wore wearable cameras for a maximum of 30 days. We used the E-Myscéal web-based lifelog retrieval system to process and analyze the wearable camera image data set. Search terms for 7 daily activities (physical activity, gardening, shopping, screen time, drinking, eating, and medication intake) were developed and used for image retrieval. Sensitivity analysis was conducted to compare the number of images retrieved using different search terms. Temporal patterns in daily activities were examined, and differences before and after hospitalization were assessed. Results E-Myscéal exhibited sensitivity to specific search terms, leading to significant variations in the number of images retrieved for each activity. The highest number of images returned were related to eating and drinking, with fewer images for physical activity, screen time, and taking medication. The majority of captured activities occurred before midday. Notably, temporal differences in daily activity patterns were observed for participants hospitalized during this study. The number of medication images increased after hospital discharge, while screen time images decreased. Conclusions Wearable cameras offer valuable insights into daily activities and self-management in people living with HF. E-Myscéal efficiently retrieves relevant images, but search term sensitivity underscores the need for careful selection.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Daily Activity Lifelogs of People With Heart Failure: Observational Study

Tegegne,

Tran,

Nourse

et al. 2024

JMIR Form Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…Figure 12 shows the time distribution for correct responses across all participating teams and evaluation topics. MySceal [53] has the best median time among the top 6 systems on the leaderboard while Memento [1] stands at 5th position in terms of median time of submission, better only to LifeSeeker's [44] tally. Figure 13 shows the count of occurences when the system was in top 3 to submit a correct response for a query.…”

Section: Analysis Of Topic 21mentioning

confidence: 99%

Memento: a prototype search engine for LSC 2021

Alam

Graham

2023

Multimed Tools Appl

View full text Add to dashboard Cite

In this extended paper, we describe our lifelog retrieval system called Memento which participated in the 2021 Lifelog Search Challenge in detail. Memento leverages semantic representations of images and textual queries projected into a common latent space to facilitate effective retrieval, aiming to bridge the existing semantic gap between complex visual scenes/events and user information needs expressed as textual and faceted queries. Our system also has a minimalist user interface which includes functionalities such as visual data filtering and temporal search. Finally, we include a comparative analysis of Memento’s performance at LSC 2021 and suggest improvements for future iterations of the system.

show abstract

“…Different lifelog benchmarking workshops/challenges have been established with distinctive evaluation metrics to assess lifelog systems, with the common objective being to facilitate the effective retrieval of specific lifelog images in an interactive or automatic manner. The standard approach taken by existing lifelog retrieval systems, such as MyScéal [26] and LifeSeeker [20], is assigning semantic context, e.g., visual concepts, to lifelog photos and applying traditional information retrieval techniques to produce a ranked list of relevant images. This approach treats each lifelog photo individually, which does not exploit the temporal and continuous nature of lifelog data.…”

Section: Lifelogs and Personal Data Analyticsmentioning

confidence: 99%

MultiMedia Modeling

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Recollecting details from lifelog data involves a higher level of granularity and reasoning than a conventional lifelog retrieval task. Investigating the task of Question Answering (QA) in lifelog data could help in human memory recollection, as well as improve traditional lifelog retrieval systems. However, there has not yet been a standardised benchmark dataset for the lifelog-based QA. In order to provide a first dataset and baseline benchmark for QA on lifelog data, we present a novel dataset, LLQA, which is an augmented 85-day lifelog collection and includes over 15,000 multiple-choice questions. We also provide different baselines for the evaluation of future works. The results showed that lifelog QA is a challenging task that requires more exploration. The dataset is publicly available at https://github.com/allie-tran/LLQA.

show abstract

Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21

Cited by 30 publications

References 32 publications

Daily Activity Lifelogs of People With Heart Failure: Observational Study

Daily Activity Lifelogs of People With Heart Failure: Observational Study

Memento: a prototype search engine for LSC 2021

MultiMedia Modeling

Contact Info

Product

Resources

About