EYEWATCHME—3D Hand and object tracking for inside out activity analysis

Li, Sun; Klank, Ulrich; Beetz, Michael

doi:10.1109/cvprw.2009.5204358

Cited by 31 publications

(25 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is common to find hierarchical methods for higher inference using as an input the shape and position of the hands. Some example applications lie in the area of activity recognition [4,5,20], user-machine interaction [6,22], or hand posture inference [23].…”

Section: State Of the Artmentioning

confidence: 99%

A Sequential Classifier for Hand Detection in the Framework of Egocentric Vision

Betancourt

2014

2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops

View full text Add to dashboard Cite

Abstract-Hand detection is one of the most explored areas in Egocentric Vision Video Analysis for wearable devices. Current methods are focused on pixel-by-pixel hand segmentation, with the implicit assumption of hand presence in almost all activities. However, this assumption is false in many applications for wearable cameras. Ignoring this fact could affect the whole performance of the device since hand measurements are usually the starting point for higher level inference, or could lead to inefficient use of computational resources and battery power. In this paper we propose a two-level sequential classifier, in which the first level, a hand-detector, deals with the possible presence of hands from a global perspective, and the second level, a hand-segmentator, delineates the hand regions at pixel level in the cases indicated by the first block. The performance of the sequential classifier is stated in probabilistic notation as a combination of both, classifiers allowing to test new handdetectors independently of the type of segmentation and the dataset used in the training stage. Experimental results show a considerable improvement in the detection of true negatives, without compromising the performance of the true positives.

show abstract

Section: State Of the Artmentioning

confidence: 99%

A Sequential Classifier for Hand Detection in the Framework of Egocentric Vision

Betancourt

2014

2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops

View full text Add to dashboard Cite

show abstract

“…We employ a similar idea but tailored to the egocentric setting. We draw inspiration from previous studies that highlight the importance of hands as salient cues towards action and activity recognition [4,7,10,38,39,40,41,42,43,44], and propose to directly modulate the density of the video sampling based on the detected hand regions. As in [45], we compare our approach with several masking schemes and feature extraction methods.…”

Section: Adaptive Sampling For Egocentric Action Recognitionmentioning

confidence: 99%

“…There are several threads of literature to cite. First, action recognition from firstperson videos is itself a relatively recent but rapidly expanding area of exploration [2,3,7,38,39,40,41,42,43,44]. As far as the authors know, all existing techniques employ uniform spatiotemporal sampling of the egocentric video.…”

Section: Adaptive Sampling For Egocentric Action Recognitionmentioning

confidence: 99%

On-the-fly hand detection training with application in egocentric action recognition

Kumar

Kyal

et al. 2015

2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

We propose a novel approach to segment hand regions in egocentric video that requires no manual labeling of training samples. The user wearing a head-mounted camera is prompted to perform a simple gesture during an initial calibration step. A combination of color and motion analysis that exploits knowledge of the expected gesture is applied on the calibration video frames to automatically label hand pixels in an unsupervised fashion. The hand pixels identified in this manner are used to train a statisticalmodel-based hand detector. Superpixel region growing is used to perform segmentation refinement and improve robustness to noise. Experiments show that our hand detection technique based on the proposed on-the-fly training approach significantly outperforms state-of-the-art techniques with respect to accuracy and robustness on a variety of challenging videos. This is due primarily to the fact that training samples are personalized to a specific user and environmental conditions. We also demonstrate the utility of our hand detection technique to inform an adaptive video sampling strategy that improves both computational speed and accuracy of egocentric action recognition algorithms. Finally, we offer an egocentric video dataset of an insulin self-injection procedure with action labels and hand masks that can serve towards future research on both hand detection and egocentric action recognition.

show abstract

“…Many researchers have proposed the use of hand tracking and gesture recognition as a powerful and natural interface for interaction with computers, game machines, and/or television sets [1]- [6]. In particular, bare hand tracking allows the user to control devices without attaching additional devices.…”

Section: Introductionmentioning

confidence: 99%

A Long-Range Touch Interface for Interaction with Smart TVs

Lee¹

2012

ETRI J

View full text Add to dashboard Cite

A powerful interaction mechanism is one of the key elements for the success of smart TVs, which demand far more complex interactions than traditional TVs. This paper proposes a novel interface based on the famous touch interaction model but utilizes long-range bare hand tracking to emulate touch actions. To satisfy the essential requirements of high accuracy and immediate response, the proposed hand tracking algorithm adopts a fast colorbased tracker but with modifications to avoid the problems inherent to those algorithms. By using online modeling and motion information, the sensitivity to the environment can be greatly decreased. Furthermore, several ideas to solve the problems often encountered by users interacting with smart TVs are proposed, resulting in a very robust hand tracking algorithm that works superbly, even for users with sleeveless clothing. In addition, the proposed algorithm runs at a very high speed of 82.73 Hz. The proposed interface is confirmed to comfortably support most touch operations, such as clicks, swipes, and drags, at a distance of three meters, which makes the proposed interface a good candidate for interaction with smart TVs. Keywords: Smart TV, touch interface, bare hand tracking, human computer interaction. Manuscript received Oct. 19, 2011; revised Aug. 14, 2012; accepted Aug. 22, 2012. Jaeyeon Lee (phone: +82 42 860 5507, leejy@etri.re.kr), DoHyung Kim (dhkim008@etri.re.kr), Jaehong Kim (jhkim504@etri.re.kr ), Jae-il Cho (jicho@etri.re.kr), and Joochan Sohn (jcsohn@etri.re.kr) are with the IT Convergence Technology Research Laboratory, ETRI, Daejeon, Rep. of Korea.http://dx

show abstract

EYEWATCHME—3D Hand and object tracking for inside out activity analysis

Cited by 31 publications

References 13 publications

A Sequential Classifier for Hand Detection in the Framework of Egocentric Vision

A Sequential Classifier for Hand Detection in the Framework of Egocentric Vision

On-the-fly hand detection training with application in egocentric action recognition

A Long-Range Touch Interface for Interaction with Smart TVs

Contact Info

Product

Resources

About