Wearable Hand Activity Recognition for Event Summarization

Mayol, W.W.; Murray, David

doi:10.1109/iswc.2005.57

Cited by 50 publications

(33 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several threads of literature to cite. First, action recognition from firstperson videos is itself a relatively recent but rapidly expanding area of exploration [2,3,7,38,39,40,41,42,43,44]. As far as the authors know, all existing techniques employ uniform spatiotemporal sampling of the egocentric video.…”

Section: Adaptive Sampling For Egocentric Action Recognitionmentioning

confidence: 99%

“…We employ a similar idea but tailored to the egocentric setting. We draw inspiration from previous studies that highlight the importance of hands as salient cues towards action and activity recognition [4,7,10,38,39,40,41,42,43,44], and propose to directly modulate the density of the video sampling based on the detected hand regions. As in [45], we compare our approach with several masking schemes and feature extraction methods.…”

Section: Adaptive Sampling For Egocentric Action Recognitionmentioning

confidence: 99%

See 1 more Smart Citation

On-the-fly hand detection training with application in egocentric action recognition

Kumar

Kyal

et al. 2015

2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

We propose a novel approach to segment hand regions in egocentric video that requires no manual labeling of training samples. The user wearing a head-mounted camera is prompted to perform a simple gesture during an initial calibration step. A combination of color and motion analysis that exploits knowledge of the expected gesture is applied on the calibration video frames to automatically label hand pixels in an unsupervised fashion. The hand pixels identified in this manner are used to train a statisticalmodel-based hand detector. Superpixel region growing is used to perform segmentation refinement and improve robustness to noise. Experiments show that our hand detection technique based on the proposed on-the-fly training approach significantly outperforms state-of-the-art techniques with respect to accuracy and robustness on a variety of challenging videos. This is due primarily to the fact that training samples are personalized to a specific user and environmental conditions. We also demonstrate the utility of our hand detection technique to inform an adaptive video sampling strategy that improves both computational speed and accuracy of egocentric action recognition algorithms. Finally, we offer an egocentric video dataset of an insulin self-injection procedure with action labels and hand masks that can serve towards future research on both hand detection and egocentric action recognition.

show abstract

Section: Adaptive Sampling For Egocentric Action Recognitionmentioning

confidence: 99%

Section: Adaptive Sampling For Egocentric Action Recognitionmentioning

confidence: 99%

On-the-fly hand detection training with application in egocentric action recognition

Kumar

Kyal

et al. 2015

2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

show abstract

“…The use of object recognition in egocentric vision systems dates back to the DyPERS system of Schiele et al [23]. Mayol and Murray [13] recognized manipulation activities leveraging skin color and histogram-based object classification from a shoulder-mounted camera. Ren and Philipose [18] collected a large database of egocentric videos of objects to facilitate research in egocentric object recognition.…”

Section: Related Workmentioning

confidence: 99%

Egocentric Object Recognition Leveraging the 3D Shape of the Grasping Hand

Lin

Hua

Mordohai

2015

Computer Vision - ECCV 2014 Workshops

View full text Add to dashboard Cite

Abstract. We present a systematic study on the relationship between the 3D shape of a hand that is about to grasp an object and recognition of the object to be grasped. In this paper, we investigate the direction from the shape of the hand to object recognition for unimpaired users. Our work shows that the 3D shape of a grasping hand from an egocentric point of view can help improve recognition of the objects being grasped. Previous work has attempted to exploit hand interactions or gaze information in the egocentric setting to guide object segmentation. However, all such analyses are conducted in 2D. We hypothesize that the 3D shape of a grasping hand is highly correlated to the physical attributes of the object being grasped. Hence, it can provide very beneficial visual information for object recognition. We validate this hypothesis by first building a 3D, egocentric vision pipeline to segment and reconstruct dense 3D point clouds of the grasping hands. Then, visual descriptors are extracted from the point cloud and subsequently fed into an object recognition system to recognize the object being grasped. Our experiments demonstrate that the 3D hand shape can indeed greatly help improve the visual recognition accuracy, when compared with the baseline where only 2D image features are utilized.

show abstract

“…According to [20], known for being the first public dataset in FPV for object recognition, hand-detection/segmentation methods can be grouped in two: model-driven and data-driven. The former uses a computerized model of the hands to recreate the image of the videos [30], while the latter exploit image features to infer about hand location, shape and position [19,27,21].…”

Section: Introductionmentioning

confidence: 99%

A Dynamic Approach and a New Dataset for Hand-detection in First Person Vision

Betancourt

Morerio

Barakova

et al. 2015

Computer Analysis of Images and Patterns

View full text Add to dashboard Cite

Abstract. Hand detection and segmentation methods stand as two of the most most prominent objectives in First Person Vision. Their popularity is mainly explained by the importance of a reliable detection and location of the hands to develop human-machine interfaces for emergent wearable cameras. Current developments have been focused on hand segmentation problems, implicitly assuming that hands are always in the field of view of the user. Existing methods are commonly presented with new datasets. However, given their implicit assumption, none of them ensure a proper composition of frames with and without hands, as the hand-detection problem requires. This paper presents a new dataset for hand-detection, carefully designed to guarantee a good balance between positive and negative frames, as well as challenging conditions such as illumination changes, hand occlusions and realistic locations. Additionally, this paper extends a state-of-the-art method using a dynamic filter to improve its detection rate. The improved performance is proposed as a baseline to be used with the dataset.

show abstract

Wearable Hand Activity Recognition for Event Summarization

Cited by 50 publications

References 17 publications

On-the-fly hand detection training with application in egocentric action recognition

On-the-fly hand detection training with application in egocentric action recognition

Egocentric Object Recognition Leveraging the 3D Shape of the Grasping Hand

A Dynamic Approach and a New Dataset for Hand-detection in First Person Vision

Contact Info

Product

Resources

About