Combining image descriptors to effectively retrieve events from visual lifelogs

Abstract

The SenseCam is a wearable camera that passively captures approximately 3,000 images per day, which equates to almost one million images per year. It is used to create a personal visual recording of the wearer's life and generates information which can be helpful as a human memory aid. For such a large amount of visual information to be of any use, it is accepted that it should be structured into "events", of which there are about 8,000 in a wearer's average year. In automatically segmenting SenseCam images into events, it will then be useful for users to locate other events similar to a given event e.g. "what other times was I walking in the park?", "show me other events when I was in a restaurant". On two datasets of 240k and 1.8M images containing topics with a variety of information needs, we evaluate the fusion of MPEG-7, SIFT, and SURF content-based retrieval techniques to address the event search issue. We have found that our proposed fusion approach of MPEG-7 and SURF offers an improvement on using either of those sources or SIFT individually, and we have also shown how a lifelog event is modeled has a large effect on the retrieval performance.

An investigation into event decay from large personal media archives

“…Our answer is that while we observed evidence of event boundary decay (only 63% overlap), we do not believe that this poses as large a challenge for maintaining a long-term lifelog as we had expected, e.g. for determining event boundaries automatically [9], determining the interestingness of events [10], or indeed in retrieving events from a lifelog [8](Section 5.3)…”

Localizing global descriptors for content-based image retrieval

“…A 'merging' fusion combined with a support vector machine (SVM) classifier, a back-propagation fusion with a KNN classifier and a Fuzzy-ART neurofuzzy network strategy are explored, which can be extended in matching the segments of an image with predefined object models. The fusion (baseline fusion and score fusion) of MPEG-7, SIFT and SURF is also explored and evaluated in [32] to address content-based event search. The detailed results conclude that the MPEG-7, SIFT and SURF are broadly comparable and also highly complementary.…”

Place Recognition via 3D Modeling for Personal Activity Lifelog Using Wearable Camera

“…Doherty et al [4] augmented streams of Lifelog images with geographic data by including locational information provided by a GPS unit rather than estimated from the visual content. Torre et al [19] recorded the actions and displacements of several subjects using fixed cameras, motion capture, inertial sensors and head-worn narrow angle cameras, that can provide precise data on the movements and ongoing instrumental activites in a restricted place.…”