Pixel-Level Hand Detection in Ego-centric Videos

Li, Cheng; Kitani, Kris M.

doi:10.1109/cvpr.2013.458

Cited by 210 publications

(250 citation statements)

References 20 publications

Supporting

Mentioning

250

Contrasting

Order By: Relevance

“…Many pixel-based hand segmentation methods have been proposed in the literature [13,14,15,16,17,18,19,20,21,22,23,24]. The feature descriptor used to perform the detection is a critical factor.…”

Section: Hand Detectionmentioning

confidence: 99%

“…These techniques may be computationally prohibitive for real-time applications, a problem that is exacerbated by the limited computational resources of mobile and wearable devices. Researchers have also explored combining color with additional features such as texture [14,16,18,23]). In [14], a generic pixel-level hand detector based on a combination of color, texture, and gradient histogram features is trained using over 600 manually labeled hand images (over 200 million labeled pixels) acquired under various illumination conditions and backgrounds, and has shown to outperform several baseline approaches.…”

Section: Hand Detectionmentioning

confidence: 99%

“…Researchers have also explored combining color with additional features such as texture [14,16,18,23]). In [14], a generic pixel-level hand detector based on a combination of color, texture, and gradient histogram features is trained using over 600 manually labeled hand images (over 200 million labeled pixels) acquired under various illumination conditions and backgrounds, and has shown to outperform several baseline approaches. This method shows improvements in detection accuracy by including additional cues to color; however, once again, this benefit is gained at the expense of computational complexity.…”

Section: Hand Detectionmentioning

confidence: 99%

See 2 more Smart Citations

On-the-fly hand detection training with application in egocentric action recognition

Kumar

Kyal

et al. 2015

2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

We propose a novel approach to segment hand regions in egocentric video that requires no manual labeling of training samples. The user wearing a head-mounted camera is prompted to perform a simple gesture during an initial calibration step. A combination of color and motion analysis that exploits knowledge of the expected gesture is applied on the calibration video frames to automatically label hand pixels in an unsupervised fashion. The hand pixels identified in this manner are used to train a statisticalmodel-based hand detector. Superpixel region growing is used to perform segmentation refinement and improve robustness to noise. Experiments show that our hand detection technique based on the proposed on-the-fly training approach significantly outperforms state-of-the-art techniques with respect to accuracy and robustness on a variety of challenging videos. This is due primarily to the fact that training samples are personalized to a specific user and environmental conditions. We also demonstrate the utility of our hand detection technique to inform an adaptive video sampling strategy that improves both computational speed and accuracy of egocentric action recognition algorithms. Finally, we offer an egocentric video dataset of an insulin self-injection procedure with action labels and hand masks that can serve towards future research on both hand detection and egocentric action recognition.

show abstract

Section: Hand Detectionmentioning

confidence: 99%

Section: Hand Detectionmentioning

confidence: 99%

Section: Hand Detectionmentioning

confidence: 99%

See 1 more Smart Citation

On-the-fly hand detection training with application in egocentric action recognition

Kumar

Kyal

et al. 2015

2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

show abstract

“…1. We first detect hands at the pixel level with [19], computing a hand probability value for each pixel on the basis of the color and texture of a local surrounding image patch. The output probabilities are averaged over recent frames to remove noise.…”

Section: Combined Motion and Shape Approachmentioning

confidence: 99%

“…Baradi et al [18] applied dense trajectories to gesture recognition in the first-person views. They introduced pixel-level hand detection [19] to reduce tracking keypoints and improved both accuracy and performance. Although they demonstrated that dense trajectories can be applied for gesture recognition in first-person view scenarios, they used only local features from dense trajectories.…”

Section: Introductionmentioning

confidence: 99%

Recognizing hand-object interactions in wearable camera videos

Ishihara

Takagi

et al. 2015

2015 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Wearable computing technologies are advancing rapidly and enabling users to easily record daily activities for applications such as life-logging or health monitoring. Recognizing hand and object interactions in these videos will help broaden application domains, but recognizing such interactions automatically remains a difficult task. Activity recognition from the first-person point-of-view is difficult because the video includes constant motion, cluttered backgrounds, and sudden changes of scenery. Recognizing hand-related activities is particularly challenging due to the many temporal and spatial variations induced by hand interactions. We present a novel approach to recognize hand-object interactions by extracting both local motion features representing the subtle movements of the hands and global hand shape features to capture grasp types. We validate our approach on multiple egocentric action datasets and show that state-of-the-art performance can be achieved by considering both local motion and global appearance information.Index Terms-Wearable cameras, first-person point-ofview, activity recognition

show abstract

3D Ego-Pose Estimation via Imitation Learning

Yuan

Kitani

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Pixel-Level Hand Detection in Ego-centric Videos

Cited by 210 publications

References 20 publications

On-the-fly hand detection training with application in egocentric action recognition

On-the-fly hand detection training with application in egocentric action recognition

Recognizing hand-object interactions in wearable camera videos

3D Ego-Pose Estimation via Imitation Learning

Contact Info

Product

Resources

About