Investigating keyframe selection methods in the novel domain of passively captured visual lifelogs

Abstract

The SenseCam is a passive capture wearable camera, worn around the neck, and when worn continuously it takes an average of 1,900 images per day. It can be used to create a personal lifelog or visual recording of the wearer's life which can be helpful as an aid to human memory. For such a large amount of visual information to be useful, it needs to be structured into "events", which can be achieved through automatic segmentation. An important component of this structuring process is the selection of keyframes to represent individual events. This work investigates a variety of techniques for the selection of a single representative keyframe image from each event, in order to provide the user with an instant visual summary of that event. In our experiments we use a large test set of 2,232 lifelog events collected by 5 users over a time period of one month each. We propose a novel keyframe selection technique which seeks to select the image with the highest "quality" as the keyframe. The inclusion of "quality" approaches in keyframe selection is demonstrated to be useful owing to the high variability in image visual quality within passively captured image collections.

LifeLogging: Personal Big Data

“…In addition, image saliency has also been shown to be important when selecting event keyframes, as discussed in Doherty et al (2008). These ideas were inspired from description of keyframe selection in the video domain by Girgensohn and Boreczky (2000), and represented an alternative to simpler techniques such as selecting the middle image in each event as proposed by Smeaton and Browne.…”

Keyframe detection in visual lifelogs

“…Details of how images are segmented into these distinct events can be found in [2]. The three scenarios investigated are known as Static Scene, Random Scene and Return Scene.…”

“…Given the low quality of SenseCam images [1], we determine the quality of each SenseCam image based on its contrast and salience properties [2]. Given an image quality score for each image we then apply the Kapur adaptive thresholding technique to select a subset of images from the event that are of a sufficiently high image quality.…”

“…In [1], SIFT descriptors are used to detect important settings in SenseCam images. In [2], MPEG-7 features were used in conjuction with sensor data to structure collections of SenseCam images into events. Given that each event typically consists of up to 80-100 images, in this paper we introduce a novel concept for selecting a representative image through the use of SIFT features.…”

“…the management of the larger volume of image data generated by such devices remains a challenging problem. In previous work, we successfully split each day's worth of images into distinct events or activities [1,2].…”