Pixel-Level Hand Detection with Shape-Aware Structured Forests

Abstract

Hand detection has many important applications in HCI, yet it is a challenging problem because the appearance of hands can vary greatly in images. In this paper, we propose a novel method for efficient pixel-level hand detection. Unlike previous method which assigns a binary label to every pixel independently, our method estimates a probability shape mask for a pixel using structured forests. This approach can better exploit hand shape information in the training data, and enforce shape constraints in the estimation. Aggregation of multiple predictions generated from neighboring pixels further improves the robustness of our method. We evaluate our method on both ego-centric videos and unconstrained still images. Experiment results show that our method can detect hands efficiently and outperform other state-of-the-art methods.

On-the-fly hand detection training with application in egocentric action recognition

“…Many pixel-based hand segmentation methods have been proposed in the literature [13,14,15,16,17,18,19,20,21,22,23,24]. The feature descriptor used to perform the detection is a critical factor.…”

“…These techniques may be computationally prohibitive for real-time applications, a problem that is exacerbated by the limited computational resources of mobile and wearable devices. Researchers have also explored combining color with additional features such as texture [14,16,18,23]). In [14], a generic pixel-level hand detector based on a combination of color, texture, and gradient histogram features is trained using over 600 manually labeled hand images (over 200 million labeled pixels) acquired under various illumination conditions and backgrounds, and has shown to outperform several baseline approaches.…”

Coarse-to-fine online learning for hand segmentation in egocentric video

“…Li and Kitani [1] predict hand pixel using color and gradient features based on Random Forest classifiers. Zhu et al [2] extend the pixel-level method by introducing shape information of pixels based on structured forests. Baraldi et al [5] utilize temporal and spatial coherence strategy to improve the hand segmentation of the pixel-level method.…”

“…Li and Kitani [1,4] propose a pixel-level hand detection method using color-and texture-based features. Zhu et al [2] propose a method which use local hand shape information in the training data and enforces shape constraints in the estimation. Serra et al [3] integrate temporal and spatial consistency to complement the appearance features.…”

“…Great efforts have been made in detecting user's hand from the egocentric video especially in pixel-level detection [1][2][3][4][5][6][7]. Most of these methods are under an implicit assumption that the hand presents in the video all the time.…”