Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data

Sridhar, Srinath; Oulasvirta, Antti; Theobalt, Christian

doi:10.1109/iccv.2013.305

Cited by 211 publications

(209 citation statements)

References 19 publications

Supporting

Mentioning

207

Contrasting

Order By: Relevance

“…However, recognizing complex hand gestures is clearly challenging from such 2D input [6]. The recent advancements in processing power and the emergence of consumer grade depth cameras, however, have enabled a number of high fidelity gestural interactive systems in HCI [10,12,21] and fine-grained 3D hand-pose estimation in real-time [19,30,35]. The current state-of-the art can be broken down into methods relying on model-fitting and temporal tracking [30,35], and those leveraging per-pixel hand part classification [19,38].…”

Section: Related Workmentioning

confidence: 99%

In-air gestures around unmodified mobile devices

Song

Sörös

Pece

et al. 2014

Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology

147

View full text Add to dashboard Cite

Figure 1: Touch input is expressive but can occlude large parts of the screen (A). We propose a machine learning based algorithm for gesture recognition expanding the interaction space around the mobile device (B), adding in-air gestures and hand-part tracking (D) to commodity off-the-shelf mobile devices, relying only on the device's camera (and no hardware modifications). We demonstrate a number of compelling interactive scenarios including bi-manual input to mapping and gaming applications (C+D). The algorithm runs in real time and can even be used on ultra-mobile devices such as smartwatches (E). ABSTRACTWe present a novel machine learning based algorithm extending the interaction space around mobile devices. The technique uses only the RGB camera now commonplace on off-the-shelf mobile devices. Our algorithm robustly recognizes a wide range of in-air gestures, supporting user variation, and varying lighting conditions. We demonstrate that our algorithm runs in real-time on unmodified mobile devices, including resource-constrained smartphones and smartwatches. Our goal is not to replace the touchscreen as primary input device, but rather to augment and enrich the existing interaction vocabulary using gestures. While touch input works well for many scenarios, we demonstrate numerous interaction tasks such as mode switches, application and task management, menu selection and certain types of navigation, where such input can be either complemented or better served by inair gestures. This removes screen real-estate issues on small touchscreens, and allows input to be expanded to the 3D space around the device. We present results for recognition accuracy (93% test and 98% train), impact of memory footprint and other model parameters. Finally, we report results from preliminary user evaluations, discuss advantages and limitations and conclude with directions for future work.

show abstract

Section: Related Workmentioning

confidence: 99%

In-air gestures around unmodified mobile devices

Song

Sörös

Pece

et al. 2014

Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology

147

View full text Add to dashboard Cite

show abstract

“…We evaluated our method on the MPI Dexter 1 hand dataset [78] in order to validate whether it can be used for matching similar motions. The dataset consists of seven sequences of hand motions of a single actor.…”

Section: Matching Similar Motionsmentioning

confidence: 99%

Learning Kinematic Structure Correspondences Using Multi-Order Similarities

Chang

Fischer

Petit

et al. 2018

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-In this paper, we present a novel framework for finding the kinematic structure correspondences between two articulated objects in videos via hypergraph matching. In contrast to appearance and graph alignment based matching methods, which have been applied among two similar static images, the proposed method finds correspondences between two dynamic kinematic structures of heterogeneous objects in videos. Thus our method allows matching the structure of objects which have similar topologies or motions, or a combination of the two. Our main contributions can be summarised as follows: (i) casting the kinematic structure correspondence problem into a hypergraph matching problem by incorporating multi-order similarities with normalising weights, (ii) introducing a structural topology similarity measure by aggregating topology constrained subgraph isomorphisms, (iii) measuring kinematic correlations between pairwise nodes, and (iv) proposing a combinatorial local motion similarity measure using geodesic distance on the Riemannian manifold. We demonstrate the robustness and accuracy of our method through a number of experiments on synthetic and real data, outperforming various other state of the art methods. Our method is not limited to a specific application nor sensor, and can be used as building block in applications such as action recognition, human motion retargeting to robots, and articulated object manipulation amongst others.

show abstract

“…For each set, we quote the number of samples, classes and subjects, provided that they are explicitly given in the database description. It may be noticed that besides our two hand gesture recognition (HGR) sets (marked as bold), there are only two data sets (ColorTip [52] and Dexter 1 [50]) with landmarks localization annotated. Moreover, in these two cases, the metadata include only the fingertips locations.…”

Section: Data Setsmentioning

confidence: 99%

Hand landmarks detection and localization in color images

Grzejszczak

Kawulok

Gałuszka

2015

Multimed Tools Appl

View full text Add to dashboard Cite

This paper introduces a new method for detecting and localizing hand landmarks in 2D color images. Location of the hand landmarks is an important source of information for recognizing hand gestures, effectively exploited in a number of recent methods which operate from the depth maps. However, this problem has not yet been satisfactorily solved for 2D color images. Here, we propose to analyze the skin-presence masks, as well as the directional image of a hand using the distance transform and template matching. This makes it possible to detect the landmarks located both at the contour and inside the hand masks. Moreover, we performed an extensive experimental study to compare the proposed method with a number of state-of-the-art algorithms. The obtained quantitative and qualitative results clearly indicate that our approach outperforms other methods, which may help improve the existing gesture recognition systems.

show abstract

Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data

Cited by 211 publications

References 19 publications

In-air gestures around unmodified mobile devices

In-air gestures around unmodified mobile devices

Learning Kinematic Structure Correspondences Using Multi-Order Similarities

Hand landmarks detection and localization in color images

Contact Info

Product

Resources

About