Hand segmentation for gesture recognition in EGO-vision

Abstract

Portable devices for first-person camera views will play a central role in future interactive systems. One necessary step for feasible human-computer guided activities is gesture recognition, preceded by a reliable hand segmentation from egocentric vision. In this work we provide a novel hand segmentation algorithm based on Random Forest superpixel classification that integrates light, time and space consistency. We also propose a gesture recognition method based Exemplar SVMs since it requires a only small set of positive sampels, hence it is well suitable for the egocentric video applications. Furthermore, this method is enhanced by using segmented images instead of full frames during test phase. Experimental results show that our hand segmentation algorithm outperforms the state-of-the-art approaches and improves the gesture recognition accuracy on both the publicly available EDSH dataset and our dataset designed for cultural heritage applications.

Exploring adoption of augmented reality smart glasses: Applications in the medical industry

“…Furthermore, gesture recognition did not emerge as an important factor during the in-depth interviews and expert panel. Developers working on gesture recognition usually focus on hand gestures (Serra et al, 2013;Lv et al, 2015), although there are alternatives such as the "gazebased interaction system" or "use of smart fabrics" (Ruminski et al, 2016). However, as the main issue is to free both hands, hand gesture recognition diminishes the hands-free feature of ARSGs; thus, voice recognition is more advantageous.…”

A Dynamic Approach and a New Dataset for Hand-detection in First Person Vision

“…Among them, hand-based methods stand as the most explored ones, aiming to exploit the conscious or unconscious hands movements for performing higher inference about the user [7] as in activity recognition [14,23] and user-machine interaction [27]. A common practice in FPV is to assume that hands are always recorded by the camera and, as a consequence, they can be located and tracked to infer more complex information.…”

“…The authors in [6] propose a characterization of the two distinct problems, namely hand-detection and hand-segmentation, and combine them in a sequential structure to improve the overall system performance. Following the definition of [6], the hand-detection level answers the yes-or-no question of the hands' presence in the frame using global features and classifiers, while the hand-segmentation level locates and outlines the hands' region in a positive frame using low level features like color under an exhaustive pixel by pixel classification framework [19,27,21].…”

“…An example is the necessity of balanced datasets in terms of hands presence as described by [6] and [8], to face the hand-detection problem under a classification framework. A balanced dataset is a realistic assumption for wearable devices and could lead to important improvements in the battery life, as well to the performance of higher-level methods like hand-based activity-recognition [12] and user-machine interaction [27]. It is worth to mentions that, as shown in section 2, existing datasets does not guarantee this condition, which makes them inappropriate to face the classification problem of the hand-detection level.…”

“…The former uses a computerized model of the hands to recreate the image of the videos [30], while the latter exploit image features to infer about hand location, shape and position [19,27,21].…”