Gesture recognition with a Time-Of-Flight camera

Kollorz, Eva; Penne, Jochen; Hornegger, Joachim; Barke, Alexander

doi:10.1504/ijista.2008.021296

Cited by 90 publications

(47 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [2] a 3D hand model is adjusted to the cloud of points obtained from the captured depth image. In [17,31,25,27,26,29] experiments, using depth sensors, are performed over static hand gestures collections, pointing out the advantages of using depth information. Another technology for obtaining range data is the one proposed in [23] where the scene is illuminated with a colored pattern, captured by a common RGB camera and later processed to infer depth information.…”

Section: Related Workmentioning

confidence: 99%

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

Molina

Pajuelo

Martínez

2015

J Sign Process Syst

View full text Add to dashboard Cite

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in: Abstract This paper presents an innovative solution based on Time-Of-Flight (TOF) video technology to motion patterns detection for real-time dynamic hand gesture recognition. The resulting system is able to detect motion-based hand gestures getting as input depth images. The recognizable motion patterns are modeled on the basis of the human arm anatomy and its degrees of freedom, generating a collection of synthetic motion patterns that is compared with the captured input patterns in order to finally classify the input gesture. For the evaluation of our system a significant collection of gestures has been compiled, getting results for 3D pattern classification as well as a comparison with the results using only 2D information.

show abstract

Section: Related Workmentioning

confidence: 99%

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

Molina

Pajuelo

Martínez

2015

J Sign Process Syst

View full text Add to dashboard Cite

show abstract

“…Several surveys have made use of this feature with various approaches to segmentation. Moreover it is possible to make use of the depth information to distinguish between ambiguous hand postures [2]. Nevertheless, it has not been possible to achieve satisfactory results utilizing only a single depth sensor.…”

Section: Related Workmentioning

confidence: 99%

Neural Network Based Data Fusion for Hand Pose Recognition with Multiple ToF Sensors

Kopinski

Gepperth

Geisler

et al. 2014

Artificial Neural Networks and Machine Learning – ICANN 2014

View full text Add to dashboard Cite

Abstract. We present a study on 3D based hand pose recognition using a new generation of low-cost time-of-flight(ToF) sensors intended for outdoor use in automotive human-machine interaction. As signal quality is impaired compared to Kinect-type sensors, we study several ways to improve performance when a large number of gesture classes is involved. We investigate the performance of different 3D descriptors, as well as the fusion of two ToF sensor streams. By basing a data fusion strategy on the fact that multilayer perceptrons can produce normalized confidences individually for each class, and similarly by designing information-theoretic online measures for assessing confidences of decisions, we show that appropriately chosen fusion strategies can improve overall performance to a very satisfactory level. Real-time capability is retained as the used 3D descriptors, the fusion strategy as well as the online confidence measures are computationally efficient.

show abstract

“…Several surveys have made use of this feature with various approaches to segmentation. Moreover it is possible to make use of the depth information to distinguish between ambiguous hand postures [3]. Nevertheless, it has not been possible to achieve satisfactory results utilising only a single depth sensor.…”

Section: Related Workmentioning

confidence: 99%

A real-time applicable 3D gesture recognition system for automobile HMI

Kopinski

Geisler

Caron

et al. 2014

17th International IEEE Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

Abstract-We present a system for 3D hand gesture recognition based on low-cost time-of-flight(ToF) sensors intended for outdoor use in automotive human-machine interaction. As signal quality is impaired compared to Kinect-type sensors, we study several ways to improve performance when a large number of gesture classes is involved. Our system fuses data coming from two ToF sensors which is used to build up a large database and subsequently train a multilayer perceptron (MLP). We demonstrate that we are able to reliably classify a set of ten hand gestures in real-time and describe the setup of the system, the utilised methods as well as possible application scenarios. I. INTRODUCTIONAs "intelligent" devices enter more and more areas of everyday life, the issue of man-machine interaction becomes ever more important. As interaction should be easy and natural for the user and also not require a high cognitive load, non-verbal means of interaction such as hand gestures will play a decisive role in this field of research. With the advent of low-cost Kinect-type 3D sensors, and more recently of low-cost ToF sensors (400-500e) that can be applied in outdoor scenarios, the use of point clouds seems a very logical choice. This presents challenges to machine learning approaches as the data dimensionality and sensor noise are high, as well as the number of interesting gesture categories. In this article we build upon earlier results [1] and demonstrate how a system can be developed and integrated into a car in order to be able to classify a gesture alphabet of ten hand poses. Our approach is purely data-driven, i.e. by extending and applying a Pointcloud descriptor to our needs we are able to set up a real-time applicable system which is robust versus daylight interferences, invariant to rotation and translation problems and moreover works without the need to formalise a possibly complicated hand model. We will first discuss the related work relevant for our research (Sec. II) and then go on to describe the setup of our system within an automobile environment. Subsequently we describe the sensors and the used database in Sec. IV. In Sec. V we go on to give an account of the used different holistic point cloud descriptors and explain the meaning of the parameter variations we will test. Sec. VI summarises the implemented NN classes and the choice of parameters. The key questions we will investigate in Sec. VII concern the generalisation

show abstract

Gesture recognition with a Time-Of-Flight camera

Cited by 90 publications

References 45 publications

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

Neural Network Based Data Fusion for Hand Pose Recognition with Multiple ToF Sensors

A real-time applicable 3D gesture recognition system for automobile HMI

Contact Info

Product

Resources

About